-
Notifications
You must be signed in to change notification settings - Fork 1.1k
GNIP 14 New Search Backend
It is proposed to add a 'new' (additional) search back-end to GeoNode that can eventually make use of a full text search engine which is capable of indexing all of the model classes in the system including Layers, Maps, Profiles and new classes such as Groups, Collections etc. This new search back-end should provide an API that can be used both by the html/js based search page as well as in by a GXP widget in the Map Composer (and by external applications).
Initial work under this proposal has been doing by Ian Schneider. After a week-long sprint ending Sept. 28, 2012, Ian and Rob Marianski worked to extract this effort from MapStory. The result is the squashed branch (two commits - one for the new functionality, the other the changes in the core) here:
https://github.com/ischneider/geonode/tree/silage
Jeffrey Johnson
TBD
For Review and Comment
Pull Request Submitted: https://github.com/GeoNode/geonode/pull/425
Currently, the integration is simply replacing the 'api' endpoints originally provided in the respective layer and maps view modules. A template is provided (at /search) that shows the combined search results in action though no new client functionality is present there.
Currently GeoNode's search for both data (layers) and maps are separated. Both use database queries directly though at the time of this new functionality, layers were using CSW. There is currently no way to search for profiles.
Several GeoNode users have complained about the quality of the search results from GeoNetwork.
There is also no way to filter search results by data type (raster, vector) or by date range etc.
Silage is a fermented forage product - basically, wet hay or other field cover - created by filling a pit and covering it. Try to find a needle in there...
It was initially proposed to implement Django Haystack http://haystacksearch.org/ as a full text search engine in GeoNodePy and connect the Layer, Map and Contact (Profile) classes to it. One objection to this approach was the requirement of an additional component. This work does not attempt to use haystack but could leverage full-text search in Postgres (but does not yet).
These API endpoints will also be used by a GXP widget in the map composer and potentially by external applications.
A variation of this (that will eventually be replaced by this work) is used in MapStory: http://mapstory.org/search
The MapStory search page noted above demonstrates the potential filtering functionality more thoroughly, though this is not covered (by any client work) in this proposal. The new map page in MapStory uses the complete, unfiltered list to populate the layer chooser (note the speed - the old WMS capabilities loader was actually timing out): http://mapstory.org/maps/new
It was determined that there was no optimal way to put a generic front-end (query parsing and view) in front of this implementation or haystack. While it might be possible, one or the other implementations would likely suffer from complexity or performance.
The query parameters and response are based on this work: https://github.com/jj0hns0n/geonode/blob/api_docs/docs/developers/api.txt#L275 but is not complete with respect to (but not limited to) fields, some query options, keywords and other facet responses.
Date ranges for layers are implemented, as are bbox queries (maps and users should be possible but Map requires the additional bbox fields from ResourceInfo).
Test coverage is currently at 90% for the module, though additional effort remains to ensure that the relevance sorting (Postgres specific SQL) is tested more thoroughly for acceptance of the algorithm.
Additional work could be done to ensure that the current results returned are what they appear to be (some tests simply assert a count as determined from visual inspection during development).
Do nothing. GeoNode's current search functionality works as is, but has severe limitations.
Remove the old search implementations from geonode/maps/views.py and geonode/layers/views.py.
Some original discussion is found on the geonode-dev list here.
https://groups.google.com/a/opengeo.org/group/geonode-dev/browse_thread/thread/a3c2a00f4ccfbd6a
New thread for this here:
https://groups.google.com/a/opengeo.org/d/topic/geonode-dev/QhoqJ0j09zA/discussion