Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Faceted search #4611

Open
mwjames opened this issue Feb 29, 2020 · 12 comments · May be fixed by #4663
Open

Faceted search #4611

mwjames opened this issue Feb 29, 2020 · 12 comments · May be fixed by #4663
Labels
new feature A new, or altered behaviour of an existing functionality that fundamentally impacts behaviour

Comments

@mwjames
Copy link
Contributor

mwjames commented Feb 29, 2020

The main task and objective of Semantic MediaWiki is to extend the MediaWiki platform with an ability to store structured information and make them retrievable either as part of a query result or as data for external federation (RDF, JSON, CSV etc,).

Another objective is to provide interfaces to help make stored data queryable and visualisable which is achieved by supporting different technical solutions including embedding results into a wikipage using the #ask parser function, building queries with the help of the advanced query builder (also known as Special:Ask), or build a bridge to MediaWiki's Special:Search using the "Extended Search" (aka. SMWSearch) integration.

Faceted search

To extend those interfaces and because Semantic MediaWiki is currently missing a vital navigational piece that allows users to easily navigate existing data with a limited preset of information, I have been working for the last week on a prototype to make a "Faceted search" to become a reality with Semantic MediaWiki (the result being shown in the screen dump).

faceted-search (2)

The "Faceted search" requires some technical tweaks but it mostly works out-of-the box by relying on the prefetch mode introduced in 3.1 for query results that minimizes the amount of SQL queries required to find data for query and selected printrequests.

Motivation

Providing such interface will give users a tool at hand to narrow down possible matches using a faceted search that should not only help advanced users but also be an attractive alternative for novice users that find the Special:Ask a bit overwhelming while the Special:SearchByProperty is only usable when one knows the property and/or value to match and comes out short when trying to browse and match.

Yes, there is something called SemanticDrilldown but this extension is hardly maintained and violates the integration with SMW in part because it uses direct access to SMW tables which is discouraged to any extension that uses SMW as backend. Using raw access to tables makes an assumption about a specific data model that may or may not be supported and in worst cases creates a strain on the wiki due to use of unoptimized SQL queries. Also, due to a possible smw_terms migration in future, accessing tables directly will no longer work as it used to be and most likely make SemanticDrilldown incompatible with any future SMW release.

Notes

I took inspiration for the visual appearance of the search from the http://projectblacklight.org/ project to help devise an interface that has been proven itself in different areas such as [0, 1, 2].

The applied approach requires an initial condition (or set of conditions) to build the basis for faceted cards (or filters) otherwise aggregating data on the fly without any query restriction would make the search unlikely to perform since we cannot assume to use (or have available) the Elasticsearch aggregation.

image

[0] https://iucat.iu.edu/catalog?q=%22france+%E2%80%8F+minist%C3%A8re+de+la+guerre%E2%80%8F%22&search_field=author_browse&search_scope=catalog&sort=title_sort+asc%2C+pub_date_sort+desc

[1] https://catalyst.library.jhu.edu/catalog?f%5Bauthor_facet%5D%5B%5D=Pope%2C+Alexander%2C+1688-1744&q=Human&range%5Bpub_date_sort%5D%5Bbegin%5D=1656&range%5Bpub_date_sort%5D%5Bend%5D=1857&search_field=subject&sort=pub_date_sort+desc%2C+title_sort+asc&suppress_spellcheck=1

[2] https://collections.nlm.nih.gov/?search_field=all_fields&sort=drep3.titleSortForm+asc
@kghbln kghbln added the new feature A new, or altered behaviour of an existing functionality that fundamentally impacts behaviour label Feb 29, 2020
This was referenced Feb 29, 2020
Merged
Merged
Merged
Merged
@JeroenDeDauw
Copy link
Member

@akuckartz bring back the 👀!

@JeroenDeDauw
Copy link
Member

This is really promising @mwjames. Is it already possible to play around with what you made so far? I'm also curious how you made the UI from a technical PoV.

@mwjames
Copy link
Contributor Author

mwjames commented Mar 1, 2020

Is it already possible to play around with what you made so far?

It resides in a local branch given that I didn't have time to cleanup the code. It is only a prototype since I needed something during the week while all the other tools didn't satisfy my need. I'm also not particularly hangup on the possibility of showing other formats, the dataTable does a decent job of integrating into the surface. At some point maybe a summary table similar to what can be seen on the mentioned faceted searches can be provided but that isn't really something I'm concerned about. Maps display could also be possibility but again nothing that concerns me for the moment or the core "faceted" support.

I'm also curious how you made the UI from a technical PoV.

Plain old HTML and some CSS which resides in mustache similar files. Some HTML is generated in PHP since that options/select are quicker done in PHP. In particular the implementation doesn't make use of any AJAX/API calls and while using vue.js (reactive update etc.) could be an option at some point it will not play any role in the initial implementation. The credo here is, simple is best and I want to keep that fairly self contained.

@m-art-in
Copy link

m-art-in commented Mar 2, 2020

These are great perspectives! The functionalities seem more extensive than Semantic Drilldown. Unfortunately I can't use this extension for a while, because it doesn't harmonize with the Elastic Store. So I hope that the faceted search in the SMW core will be successful.

@jaideraf
Copy link
Member

jaideraf commented Mar 3, 2020 •
edited

Amazing! I am so excited to see it live. 😍
Congratulations, it will be very useful. As a suggestion, maybe a little triangule (like ▸ and ▾) [0] instead the [Collapse] and [Expand] labels could be used to shrink the texts.

[0] https://alligator.io/css/collapsible/

@mwjames
Copy link
Contributor Author

mwjames commented Mar 7, 2020

As a suggestion, maybe a little triangule (like ▸ and ▾) [0] instead the [Collapse] and [Expand] labels could be used shrink the texts.

We are using mw-collapsible [0] from MW here and while you could use data-collapsetext and data-expandtext to override the text, it causes a FOUC [1] and seeing those unstyled content for a blink of an eye is a no-go. Other solutions could be used but that isn't really something I concern myself with since the solution provided is available out of the box without having to reinvent the wheels. So, unless someone at the WMF fixes the issue, the text remains as provided.

fs-fouc

[0] https://www.mediawiki.org/wiki/Manual:Collapsible_elements
[1] https://en.wikipedia.org/wiki/Flash_of_unstyled_content

This was referenced Mar 7, 2020
Merged
Merged
@mwjames
Copy link
Contributor Author

mwjames commented Mar 8, 2020

I found a couple of hours to define the general direction for the interface.

Profile management

Making the interface versatile for power users yet keeping it simple for novice users is made possible by the FACETEDSEARCH_PROFILE_SCHEMA which allows to maintain different (structured) profiles so that a default profile shows only a limited set of options such as:

image

While a different profile may provide a more selective search or different filter types.

image

A profile can also define a list of exploratory queries to be displayed on an empty search input to help users quickly find some important collections.

image

A user is able to select a profile preference so that the preferred profile is selected every time s(he) uses the search.

image

The profile schema

image

The profile is validated against a JSON schema so if something doen't fit the schema then the user is informed about it such as:

image

Mobile ready

The interface uses the flexbox (CSS) system which means it is mobile ready out of the box.

image

This was referenced Mar 21, 2020
Merged
Merged
Merged
Open
@kghbln kghbln added this to the SMW 3.2.0 milestone Mar 22, 2020
@gesinn-it-gea
Copy link
Member

@mwjames this looks already very promising! Also you did a super job explaining your ideas! Thx for this.

@gesinn-it-gea
Copy link
Member

gesinn-it-gea commented Mar 29, 2020 •
edited

@mwjames would it be possible

  1. to transclude the Special page? [1]
  2. to preset filter values via URL params? UPDATE: seems to be possible, according to Add SpecialFacetedSearch, refs 4611 #4663

[1] https://www.mediawiki.org/wiki/Transclusion#Special_pages

@akuckartz akuckartz mentioned this issue May 24, 2020
Open
@akuckartz akuckartz modified the milestones: SMW 3.2.0, SMW 3.3.0 Aug 9, 2020
@kghbln kghbln modified the milestones: SMW 3.2.0, SMW 3.3.0 Sep 8, 2020
@octfx
Copy link
Contributor

octfx commented Nov 28, 2021

For others interested in this there exists Extension:WSSearch which implements this (in part?).
It requires Elasticsearch though

@jaideraf
Copy link
Member

@octfx It would be nice if we can translate the extension WSSearchFront in https://translatewiki.net. Also, to report issues we must register at https://gitlab.wikibase.nl, right?

@octfx
Copy link
Contributor

octfx commented Nov 29, 2021

Ah sorry, I should have mentioned that I'm not affiliated with Wikibase. 😃
I saw this issue by chance and noticed that WSSearch was finally released (it was introduced at EMWCon 2021)

@kghbln kghbln removed this from the SMW 4.0.0 milestone Jan 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
new feature A new, or altered behaviour of an existing functionality that fundamentally impacts behaviour
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add SpecialFacetedSearch, refs 4611 SemanticMediaWiki/SemanticMediaWiki
8 participants
@JeroenDeDauw @akuckartz @kghbln @mwjames @jaideraf @octfx @gesinn-it-gea @m-art-in