GH-1778: List graphs without count in Fuseki UI #1792

nichtich · 2023-03-09T09:14:05Z

Modified the Fuseki UI to show a list of graphs without triple counts in /edit view by default because counting all graphs is slow and the /edit interface becomes unusable for large numbers for graphs & triples.

GitHub issue resolved #1778

Tests are included
Documentation change and updates are provided for the Apache Jena website
Commits have been squashed to remove intermediate development commit messages.
Key commit messages start with the issue number (GH-xxxx or JENA-xxxx)

By submitting this pull request, I acknowledge that I am making a contribution to the Apache Software Foundation under the terms and conditions of the Contributor's Agreement.

See the Apache Jena "Contributing" guide.

kinow

Are you planning to add tests, @nichtich ? WDYT of this change @afs? I rarely use this feature, TBH. If it's OK to drop the counts, we can go ahead and finish the change with tests & docs.

nichtich · 2023-03-17T07:11:13Z

I created no tests because the current UI was not covered by tests anyway. Same applies to documentation for this feature, it just does not apply to this pull request, so I did not check both checkboxes.

kinow · 2023-03-17T07:17:01Z

Got it. Docs are all good. First we migrated the old JQuery and Bootstrap.js code to Vue.js and then added initial unit and e2e tests. Future changes after that are expected to have tests (recent previous pull requests to the UI probably have examples of that).

But let's wait for others first to see what they think of this change. And writing these tests can be a bit boring if uou are not familiar with the test harness and tooling. In tuat case just let us know and we can send a diff or commit with some tests to this PR 👍

rvesse

@kinow Some tests around this would be nice, I tried this feature the other day exploring a dataset and wasn't clear to me that it works at all. I think not counting triples by default is a good behavioural change as it doesn't scale well for large datasets.

Even just listing graphs may not scale well in all cases e.g. #1655

Modified the Fuseki UI to show a list of graphs without triple counts in /edit view by default because counting all graphs is slow and the /edit interface becomes unusable for large numbers for graphs & triples.

nichtich · 2023-03-17T11:02:14Z

I tried to add a basic test (force-push to my fork) but not having dug into Vue testing before I'm stuck with

[Vue warn]: Failed to resolve component: router-link

and I'm not sure whether the services watcher is actually called in the testing environment and what value it should have.

kinow · 2023-03-17T11:56:02Z

I tried to add a basic test (force-push to my fork) but not having dug into Vue testing before I'm stuck with

No worries @nichtich , thanks for providing the initial structure. It's a bit annoying to write these tests without having an existing test to copy (as you noticed, things are not very intuitive, and the tests being added later means you have nothing to base your work on). I can finish the test and ping you if I need help to test/cover some part of your code 👍

Thanks!

kinow

Change looks good to me. Will suggest a few modifications in the code (in a separate commit) and then we can discuss after that. @nichtich thanks for the PR!

kinow · 2023-03-26T22:37:22Z

jena-fuseki2/jena-fuseki-ui/src/views/dataset/Edit.vue

+        } else {
+          const graphs = await this.$fusekiService.listGraphs(this.datasetName, endpoint)
+          this.graphs = Object.fromEntries(graphs.map(name => [name, ""]))
+        }


I will try to split this function into two later 👍

Yeah, it looks like one function with a bool=true returns just the count, and for bool=false the graphs (with the counts too?). I think one is bringing the count to display the graphs and the count. The other is bringing the graphs and services available. If so, we can probably split this function into two, one for the graph services, and another for the counts. (Need to confirm)

kinow · 2023-03-26T22:38:30Z

jena-fuseki2/jena-fuseki-ui/src/views/dataset/Edit.vue

      this.loadingGraphs = true
      this.loadingGraph = true
      this.code = ''
      this.selectedGraph = ''
      try {
-        this.graphs = await this.$fusekiService.countGraphsTriples(this.datasetName, this.services.query['srv.endpoints'][0])
+        const endpoint = this.services.query['srv.endpoints'][0]


#1679 has some changes for the FusekiService and the src.endpoints. Once that one is merged, I will rebase here and update the code to that calls $fusekiService methods.

kinow · 2024-03-10T14:35:08Z

I noticed that most PR's were Fuseki-UI, blocked by my. I am sorry 🙇 Looking at this one now.

kinow · 2024-03-10T14:42:47Z

(PR was opened from gbv/ without allowing contributors to modify it, so I cannot rebase it or add commits with tests - ping @nichtich )

kinow · 2024-03-10T15:00:26Z

jena-fuseki2/jena-fuseki-ui/src/services/fuseki.service.js

+        query: DATASET_GRAPHS
+      }
+    })
+    return ["default", ...result.data.results.bindings.map(binding => binding.g.value)]


Cleaner than the other methods doing the forEach! Will apply that to the rest of the code in a future PR. Thanks!

kinow · 2024-03-10T15:00:45Z

jena-fuseki2/jena-fuseki-ui/src/services/fuseki.service.js

@@ -22,6 +22,7 @@ import { BUS } from '@/events'

 const DATASET_SIZE_QUERY_1 = 'select (count(*) as ?count) {?s ?p ?o}'
 const DATASET_SIZE_QUERY_2 = 'select ?g (count(*) as ?count) {graph ?g {?s ?p ?o}} group by ?g'
+const DATASET_GRAPHS = 'select ?g {graph ?g {}}'


Maybe we can rename this to $same_QUERY...

kinow · 2024-03-10T15:31:52Z

jena-fuseki2/jena-fuseki-ui/src/views/dataset/Edit.vue

+        } else {
+          const graphs = await this.$fusekiService.listGraphs(this.datasetName, endpoint)
+          this.graphs = Object.fromEntries(graphs.map(name => [name, ""]))
+        }


Yeah, it looks like one function with a bool=true returns just the count, and for bool=false the graphs (with the counts too?). I think one is bringing the count to display the graphs and the count. The other is bringing the graphs and services available. If so, we can probably split this function into two, one for the graph services, and another for the counts. (Need to confirm)

kinow · 2024-03-10T15:53:45Z

jena-fuseki2/jena-fuseki-ui/tests/unit/views/dataset/edit.vue.spec.js

+    expect(component.find('.card-body h3').text()).to.equal('Available Graphs')
+
+    // TODO: should return 1 instead
+    // expect(component.findAll('tr').length).to.equal(1)


afs · 2024-03-10T16:06:24Z

+1 to listing the graph without counts.

kinow · 2024-03-10T20:15:41Z

Alright, I have a local version that contains the function split into two, and I think I understood more about the goal of this issue (also spotted that I didn't add pagination to Edit, only to Info view).

@afs there are a few things to confirm on this PR.

So this PR disables the COUNT(*) queries for the graphs by default
It eagerly loads the list of graphs once you open the Edit view (even though we still have the button to count graphs)
- What would be the best?
  - a. load all graphs by default, without the count once the page is loaded, wait for user to click to count all graphs?
  - b. load all graphs by default, without the count once the page is loaded, do not let users click on count (go to Info view instead)
  - c. load nothing by default, wait for user to click and then load graphs without count
Even though we are removing the COUNT((*), that's still required... when you edit the graph, the UI uses that COUNT to compare against MAX_EDITABLE_SIZE, set in the code to 10000 (from old Fuseki UI)
- Should we let users try to save very large graphs and possibly overload UI and/or backend? i.e. drop the limit/need for count
- Or should we load the count when the user tries to save it (or when s/he clicks on it?

Some screenshots from the current version in this PR:

(when you open the Edit view, it queries the backend when the page is loaded)

(when you click count triples, it's the same as main branch)

kinow · 2024-03-10T20:16:38Z

I think this PR is pending some more discussion, and not yet ready to be merged even adding the tests. But good progress today 👋

kinow reviewed Mar 16, 2023

View reviewed changes

rvesse reviewed Mar 17, 2023

View reviewed changes

apacheGH-1778: List graphs without count in Fuseki UI

91d9bc5

Modified the Fuseki UI to show a list of graphs without triple counts in /edit view by default because counting all graphs is slow and the /edit interface becomes unusable for large numbers for graphs & triples.

nichtich force-pushed the issue-1778 branch from 2288086 to 91d9bc5 Compare March 17, 2023 10:53

kinow added the Fuseki UI label Mar 25, 2023

kinow reviewed Mar 26, 2023

View reviewed changes

kinow reviewed Mar 10, 2024

View reviewed changes

kinow marked this pull request as draft March 26, 2024 21:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-1778: List graphs without count in Fuseki UI #1792

GH-1778: List graphs without count in Fuseki UI #1792

nichtich commented Mar 9, 2023

kinow left a comment •

edited

nichtich commented Mar 17, 2023

kinow commented Mar 17, 2023 •

edited

rvesse left a comment

nichtich commented Mar 17, 2023

kinow commented Mar 17, 2023

kinow left a comment

kinow Mar 26, 2023

kinow Mar 10, 2024

kinow Mar 26, 2023

kinow commented Mar 10, 2024

kinow commented Mar 10, 2024 •

edited

kinow Mar 10, 2024

kinow Mar 10, 2024

kinow Mar 10, 2024

kinow Mar 10, 2024

afs commented Mar 10, 2024

kinow commented Mar 10, 2024

kinow commented Mar 10, 2024

GH-1778: List graphs without count in Fuseki UI #1792

Are you sure you want to change the base?

GH-1778: List graphs without count in Fuseki UI #1792

Conversation

nichtich commented Mar 9, 2023

kinow left a comment • edited

Choose a reason for hiding this comment

nichtich commented Mar 17, 2023

kinow commented Mar 17, 2023 • edited

rvesse left a comment

Choose a reason for hiding this comment

nichtich commented Mar 17, 2023

kinow commented Mar 17, 2023

kinow left a comment

Choose a reason for hiding this comment

kinow Mar 26, 2023

Choose a reason for hiding this comment

kinow Mar 10, 2024

Choose a reason for hiding this comment

kinow Mar 26, 2023

Choose a reason for hiding this comment

kinow commented Mar 10, 2024

kinow commented Mar 10, 2024 • edited

kinow Mar 10, 2024

Choose a reason for hiding this comment

kinow Mar 10, 2024

Choose a reason for hiding this comment

kinow Mar 10, 2024

Choose a reason for hiding this comment

kinow Mar 10, 2024

Choose a reason for hiding this comment

afs commented Mar 10, 2024

kinow commented Mar 10, 2024

kinow commented Mar 10, 2024

kinow left a comment •

edited

kinow commented Mar 17, 2023 •

edited

kinow commented Mar 10, 2024 •

edited