Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add configuration option equivalent to having "@skip kg:select" annotation in all queries #183

Open
tokarenko opened this issue Apr 23, 2024 · 4 comments
Open

Add configuration option equivalent to having "@skip kg:select" annotation in all queries #183

tokarenko opened this issue Apr 23, 2024 · 4 comments

Comments

@tokarenko
Copy link

Issue Description:

Source selection with bind is not disabled by 'FEDERATE_JOIN = false' in corese-properties.properties. It worked before as advised in #152. We are unable to get it working in new environment.

Bug Details:

Federated query is issued with BIND and EXISTS: select * \nwhere {\n bind (exists {select * \n where {\n http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .\n }\n limit 1 } as ?b0)\n}\nlimit 1000

Steps to Reproduce:

  1. Run Corese in docker with mounted corese-properties.properties containing FEDERATE_JOIN = false:
    docker run -d --rm --name CORESE -p 8095:8080 -v /home/kyzauver/corese/config:/usr/local/corese/config -v /home/kyzauver/corese/data:/usr/local/corese/data -v /home/kyzauver/corese/log:/usr/local/corese/log wimmics/corese
  2. Run federated query:
    SELECT * WHERE {
    http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o
    }

Expected Behavior:

Query is issued without BIND and EXISTS.

Actual Behavior:

Query is issued with BIND and EXISTS:

2024-04-23 06:45:05 INFO webservice.SPARQLRestAPI.getTriplesXMLForGet:311 [] - getTriplesXMLForGet
2024-04-23 06:45:05 INFO webservice.SPARQLResult.getResultFormat:90 [] - Endpoint URL: http://10.112.17.134:8095/myfederation/federate
2024-04-23 06:45:05 INFO webservice.SPARQLResult.getResultFormat:96 [] - Query: SELECT * WHERE {
http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o
}

2024-04-23 06:45:05 INFO webservice.EventManager.log:72 [] - Workflow Context:
federate : true
st:remoteHost : "10.112.31.71"
st:service : "http://ns.inria.fr/sparql-template/user"
mode : "("federate" )"^^dt:list
operation : "federate"
request : "[org.eclipse.jetty.server.Request:Request(GET http://10.112.17.134:8095/myfederation/federate?query=SELECT%20*%20WHERE%20%7B%0A%20%20%3Chttp%3A%2F%2Ftest.domain%2Fdevice.owl%23core1_Gi0%2F1_ipv4_address%3E%20%3Fp%20%3Fo%0A%7D%0A)@378a0b87]"^^dt:pointer
url : http://10.112.17.134:8095/myfederation/federate
user query: true
level: PRIVATE

2024-04-23 06:45:05 INFO webservice.EventManager.log:73 [] - {st:count="[Map: size=1]"^^dt:map, st:date="[Map: size=1]"^^dt:map, st:host="[Map: size=1]"^^dt:map, st:hostlite="[Map: size=1]"^^dt:map}
2024-04-23 06:45:05 INFO webservice.EventManager.log:74 [] - {"http://ns.inria.fr/sparql-template/user"=2}
2024-04-23 06:45:05 INFO webservice.EventManager.log:76 [] - {"10.112.31.71"=2}
2024-04-23 06:45:05 INFO webservice.TripleStore.federate:396 [] - metadata: Metadata:
@federation : [http://10.112.17.134:8095/myfederation/federate]

2024-04-23 06:45:05 INFO federate.FederateVisitor.option:447 [] - Metadata:
@federation : [http://10.112.17.134:8095/myfederation/federate]

2024-04-23 06:45:05 INFO query.ProviderService.getTimeout:919 [] - Timeout: 200000
2024-04-23 06:45:05 INFO load.Service.basicPost:253 [] - Post http://10.112.17.134:8083
2024-04-23 06:45:05 INFO load.Service.basicPost:253 [] - Post http://10.112.17.134:8225
2024-04-23 06:45:05 INFO load.Service.basicPost:286 [] - Response status: 400
2024-04-23 06:45:05 ERROR load.Service.basicPost:318 [] - http://10.112.17.134:8225
2024-04-23 06:45:05 ERROR load.Service.basicPost:319 [] - jakarta.ws.rs.client.ResponseProcessingException {
"exception": "Invalid SPARQL query: Built-in function "exists{selectwhere{http://test.domain/device.owl#core1_gi0/1_ipv4_address?p?o.}limit1}" not yet implemented; if you need it, just add it to SparqlQleverVisitor.cpp::visitTypesafe(Parser::BuiltInCallContext following the already implemented functions there",
"metadata": {
"line": 3,
"positionInLine": 8,
"query": "select * \nwhere {\n bind (exists {select * \n where {\n http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .\n }\n limit 1 } as ?b0)\n}\nlimit 1000 ",
"startIndex": 26,
"stopIndex": 140
},
"query": "select * \nwhere {\n bind (exists {select * \n where {\n http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .\n }\n limit 1 } as ?b0)\n}\nlimit 1000 ",
"resultsize": 0,
"status": "ERROR",
"time": {
"computeResult": 10,
"total": 10
}
}
2024-04-23 06:45:05 ERROR query.ProviderService.exception:507 [] - jakarta.ws.rs.client.ResponseProcessingException http://10.112.17.134:8225
2024-04-23 06:45:05 ERROR query.ProviderService.error:596 [] - service error: http://10.112.17.134:8225
2024-04-23 06:45:05 ERROR query.ProviderService.error:600 [] - {
"exception": "Invalid SPARQL query: Built-in function "exists{selectwhere{http://test.domain/device.owl#core1_gi0/1_ipv4_address?p?o.}limit1}" not yet implemented; if you need it, just add it to SparqlQleverVisitor.cpp::visitTypesafe(Parser::BuiltInCallContext following the already implemented functions there",
"metadata": {
"line": 3,
"positionInLine": 8,
"query": "select * \nwhere {\n bind (exists {select * \n where {\n http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .\n }\n limit 1 } as ?b0)\n}\nlimit 1000 ",
"startIndex": 26,
"stopIndex": 140
},
"query": "select * \nwhere {\n bind (exists {select * \n where {\n http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .\n }\n limit 1 } as ?b0)\n}\nlimit 1000 ",
"resultsize": 0,
"status": "ERROR",
"time": {
"computeResult": 10,
"total": 10
}
}
2024-04-23 06:45:05 ERROR query.ProviderService.error:602 [] - select *
where {
bind (exists {select *
where {
http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .
}
limit 1 } as ?b0)
}
limit 1000
2024-04-23 06:45:05 INFO load.Service.basicPost:286 [] - Response status: 200
2024-04-23 06:45:05 ERROR datatype.DatatypeMap.createLiteral:667 [] - Datatype error: http://www.w3.org/2001/XMLSchema#boolean
2024-04-23 06:45:05 INFO federate.Selector.process11:243 [] - Source Selection Join Test Success: true
2024-04-23 06:45:05 ERROR federate.FederateVisitor.error:977 [] - rewrite triple
2024-04-23 06:45:05 ERROR federate.FederateVisitor.error:982 [] - Undefined triple: http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .
2024-04-23 06:45:05 ERROR federate.FederateVisitor.error:316 [] - Query rewrite fail due to undefined triple

corese-properties.properties seem to load fine and contains 'FEDERATE_JOIN = false' (checked the file inside the container).

2024-04-23 06:44:08 INFO webservice.EmbeddedJettyServer.main:115 [] - Loading log4j configuration: file:/usr/local/corese/config/log4j2.xml
2024-04-23 06:44:08 INFO webservice.EmbeddedJettyServer.main:116 [] - To override log4j configuration add JVM option: -Dlog4j.configurationFile=file:/home/.../your_log4j2.xml
2024-04-23 06:44:08 INFO core.Corese.init:14 [] - Corese, Inria: Tue Apr 23 06:44:08 UTC 2024
2024-04-23 06:44:08 INFO webservice.EmbeddedJettyServer.main:231 [] - Profile: file:///usr/local/corese/config/corese-profile.ttl
2024-04-23 06:44:08 INFO webservice.EmbeddedJettyServer.main:282 [] - init = /usr/local/corese/config/corese-properties.properties
2024-04-23 06:44:08 INFO util.Property.defineFederation:967 [] - federation: /usr/local/corese/data/federation.ttl
federation: http://10.112.17.134:8095/myfederation/federate : "(http://10.112.17.134:8083/ http://10.112.17.134:8225/ )"^^dt:list
access: http://10.112.17.134:8083/
access: http://10.112.17.134:8225/
@ocorby
Copy link
Contributor

ocorby commented Apr 23, 2024 via email

@tokarenko
Copy link
Author

Thank you, Olivier, for your quick reply! @Skip kg:select does not work as expected. Query still contains bind (exists {} as var). I would be grateful if you could resolve the issue.

2024-04-24 07:57:05 INFO  webservice.SPARQLRestAPI.getTriplesXMLForGet:311 [] - getTriplesXMLForGet
2024-04-24 07:57:05 INFO  webservice.SPARQLResult.getResultFormat:90 [] - Endpoint URL: http://10.112.17.134:8095/myfederation/federate
2024-04-24 07:57:05 INFO  webservice.SPARQLResult.getResultFormat:96 [] - Query: prefix onto: <http://test.domain/device.owl#> 
@Skip kg:select
SELECT ?ip WHERE {
  <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip
}
2024-04-24 07:57:05 INFO  webservice.EventManager.log:72 [] - Workflow Context:
federate : true
st:remoteHost : "10.112.31.71"
st:service : "http://ns.inria.fr/sparql-template/user"
mode : "("federate" )"^^dt:list
operation : "federate"
request : "[org.eclipse.jetty.server.Request:Request(GET [http://10.112.17.134:8095/myfederation/federate?query=prefix%20onto%3A%20%3Chttp%3A%2F%2Ftest.domain%2Fdevice.owl%23%3E%20%0A%40Skip%20kg%3Aselect%0ASELECT%20%3Fip%20WHERE%20%7B%0A%20%20%3Chttp%3A%2F%2Ftest.domain%2Fdevice.owl%23core1_Gi0%2F1_ipv4_address%3E%20onto%3AIP%20%3Fip%0A%7D)@44b1454]"^^dt:pointer](http://10.112.17.134:8095/myfederation/federate?query=prefix%20onto%3A%20%3Chttp%3A%2F%2Ftest.domain%2Fdevice.owl%23%3E%20%0A%40Skip%20kg%3Aselect%0ASELECT%20%3Fip%20WHERE%20%7B%0A%20%20%3Chttp%3A%2F%2Ftest.domain%2Fdevice.owl%23core1_Gi0%2F1_ipv4_address%3E%20onto%3AIP%20%3Fip%0A%7D)@44b1454%5d%22%5e%5edt:pointer)
url : <http://10.112.17.134:8095/myfederation/federate>
user query: true
level: PRIVATE

2024-04-24 07:57:05 INFO  webservice.EventManager.log:73 [] - {st:count="[Map: size=1]"^^dt:map, st:date="[Map: size=1]"^^dt:map, st:host="[Map: size=1]"^^dt:map, st:hostlite="[Map: size=1]"^^dt:map}
2024-04-24 07:57:05 INFO  webservice.EventManager.log:74 [] - {"http://ns.inria.fr/sparql-template/user"=3}
2024-04-24 07:57:05 INFO  webservice.EventManager.log:76 [] - {"10.112.31.71"=3}
2024-04-24 07:57:05 INFO  webservice.TripleStore.federate:396 [] - metadata: Metadata:
@Skip : [http://ns.inria.fr/corese/kgram/select]
@federation : [http://10.112.17.134:8095/myfederation/federate]

2024-04-24 07:57:05 INFO  federate.FederateVisitor.option:447 [] - Metadata:
@Skip : [http://ns.inria.fr/corese/kgram/select]
@federation : [http://10.112.17.134:8095/myfederation/federate]

2024-04-24 07:57:05 INFO  query.ProviderService.getTimeout:919 [] - Timeout: 200000
2024-04-24 07:57:05 INFO  load.Service.basicPost:253 [] - Post http://10.112.17.134:8224/
2024-04-24 07:57:05 INFO  load.Service.basicPost:286 [] - Response status: 400
2024-04-24 07:57:05 ERROR load.Service.basicPost:318 [] - http://10.112.17.134:8224/
2024-04-24 07:57:05 ERROR load.Service.basicPost:319 [] - jakarta.ws.rs.client.ResponseProcessingException {
    "exception": "Invalid SPARQL query: Built-in function \"exists{select*where{?sonto:ip?o.}limit1}\"  not yet implemented; if you need it, just add it to SparqlQleverVisitor.cpp::visitTypesafe(Parser::BuiltInCallContext following the already implemented functions there",
    "metadata": {
        "line": 5,
        "positionInLine": 8,
        "query": "prefix kg: <http://ns.inria.fr/corese/kgram/>\nprefix onto: <http://test.domain/device.owl#>\nselect * \nwhere {\n  bind (exists {select * \n    where {\n      ?s onto:IP ?o .\n    }\n    limit 1 } as ?b0)\n  bind (exists {select * \n    where {\n      <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip .\n    }\n    limit 1 } as ?b1)\n}\nlimit 1000 ",
        "startIndex": 113,
        "stopIndex": 183
    },
    "query": "prefix kg: <http://ns.inria.fr/corese/kgram/>\nprefix onto: <http://test.domain/device.owl#>\nselect * \nwhere {\n  bind (exists {select * \n    where {\n      ?s onto:IP ?o .\n    }\n    limit 1 } as ?b0)\n  bind (exists {select * \n    where {\n      <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip .\n    }\n    limit 1 } as ?b1)\n}\nlimit 1000 ",
    "resultsize": 0,
    "status": "ERROR",
    "time": {
        "computeResult": 0,
        "total": 0
    }
}
2024-04-24 07:57:05 ERROR query.ProviderService.exception:507 [] - jakarta.ws.rs.client.ResponseProcessingException http://10.112.17.134:8224/
2024-04-24 07:57:05 ERROR query.ProviderService.error:596 [] - service error: http://10.112.17.134:8224/
2024-04-24 07:57:05 ERROR query.ProviderService.error:598 [] - {
    "exception": "Invalid SPARQL query: Built-in function \"exists{select*where{?sonto:ip?o.}limit1}\"  not yet implemented; if you need it, just add it to SparqlQleverVisitor.cpp::visitTypesafe(Parser::BuiltInCallContext following the already implemented functions there",
    "metadata": {
        "line": 5,
        "positionInLine": 8,
        "query": "prefix kg: <http://ns.inria.fr/corese/kgram/>\nprefix onto: <http://test.domain/device.owl#>\nselect * \nwhere {\n  bind (exists {select * \n    where {\n      ?s onto:IP ?o .\n    }\n    limit 1 } as ?b0)\n  bind (exists {select * \n    where {\n      <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip .\n    }\n    limit 1 } as ?b1)\n}\nlimit 1000 ",
        "startIndex": 113,
        "stopIndex": 183
    },
    "query": "prefix kg: <http://ns.inria.fr/corese/kgram/>\nprefix onto: <http://test.domain/device.owl#>\nselect * \nwhere {\n  bind (exists {select * \n    where {\n      ?s onto:IP ?o .\n    }\n    limit 1 } as ?b0)\n  b
2024-04-24 07:57:05 ERROR query.ProviderService.error:602 [] - prefix kg: <http://ns.inria.fr/corese/kgram/>
prefix onto: <http://test.domain/device.owl#>
select * 
where {
  bind (exists {select * 
    where {
      ?s onto:IP ?o .
    }
    limit 1 } as ?b0)
  bind (exists {select * 
    where {
      <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip .
    }
    limit 1 } as ?b1)
}
limit 1000 
2024-04-24 07:57:05 INFO  federate.Selector.process11:243 [] - Source Selection Join Test Success: true
2024-04-24 07:57:05 ERROR federate.FederateVisitor.error:977 [] - rewrite triple
2024-04-24 07:57:05 ERROR federate.FederateVisitor.error:982 [] - Undefined triple: <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip .
2024-04-24 07:57:05 ERROR federate.FederateVisitor.error:316 [] - Query rewrite fail due to undefined triple

@tokarenko tokarenko changed the title Source selection with bind is not disabled by FEDERATE_JOIN = false Annotation "@Skip kg:select" does not disable BIND (EXISTS{} AS var) Apr 24, 2024
@ocorby
Copy link
Contributor

ocorby commented Apr 24, 2024 via email

@tokarenko
Copy link
Author

It works as expected with lower case @Skip kg:select ! Thank you. I should have tried it before posting. I am sorry for this.

I would be grateful if you could add corresponding setting into corese-properties.properties. I would be nice to have this option set by default for all queries.

@tokarenko tokarenko changed the title Annotation "@Skip kg:select" does not disable BIND (EXISTS{} AS var) Add configuration option equivalent to having "@skip kg:select" annotation in all queries Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tokarenko @ocorby