Fix catch error on index #327
Open
+3
−2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When datasets have the wrong geospatial coverage value, they aren’t indexed so users won’t know of them and can’t update/ delete them:
ckan search-index rebuild -ei 2024-03-13 02:12:13,066 INFO [ckan.cli] Using configuration file /etc/ckan/default/production.ini 2024-03-13 02:12:13,067 INFO [ckan.config.environment] Loading static files from public-bs3 2024-03-13 02:12:13,267 WARNI [ckan.common] Option ckan.plugins is not declared 2024-03-13 02:12:13,494 INFO [ckan.config.environment] Loading templates from /usr/lib/ckan/default/src/ckan/ckan/templates-bs3 2024-03-13 02:12:13,714 WARNI [ckan.common] Option ckanext.xloader.max_content_length is not declared 2024-03-13 02:12:13,714 WARNI [ckan.common] Option ckanext.xloader.max_type_guessing_length is not declared 2024-03-13 02:12:13,714 WARNI [ckan.common] Option ckanext.xloader.max_excerpt_lines is not declared 2024-03-13 02:12:14,704 WARNI [ckan.common] Option lang is not declared 2024-03-13 02:12:14,782 WARNI [ckanext.reclineview.plugin] The Recline-based views are deprecated andwill be removed in future versions 2024-03-13 02:12:14,825 INFO [ckan.config.environment] Loading templates from /usr/lib/ckan/default/src/ckan/ckan/templates-bs3 2024-03-13 02:12:14,904 DEBUG [ckanext.harvest.model] Harvest tables defined in memory 2024-03-13 02:12:14,908 DEBUG [ckanext.harvest.model] Harvest tables already exist 2024-03-13 02:12:14,990 WARNI [ckan.common] Option ckanext_jmh.theme_name is not declared 2024-03-13 02:12:15,211 WARNI [ckan.config.middleware.flask_app] Extensions are excluded from CSRF protection! We allow extensions to run without CSRF protection but it will be forced future releases. Read the documentation for more information on how to add CSRF protection to your extension. 2024-03-13 02:12:15,301 INFO [ckan.lib.search] Rebuilding search index... 2024-03-13 02:12:15,306 INFO [ckan.lib.search] Rebuilding the whole index... Indexing dataset 1/32024-03-13 02:12:15,335 ERROR [ckan.lib.search] Error while indexing dataset ffaedeec-6345-4925-920f-da60ef617e48: TypeError("'float' object is not iterable") 2024-03-13 02:12:15,346 ERROR [ckan.lib.search] Traceback (most recent call last): File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/__init__.py", line 253, in rebuild package_index.update_dict( File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/index.py", line 106, in update_dict self.index_package(pkg_dict, defer_commit) File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/index.py", line 290, in index_package pkg_dict = item.before_dataset_index(pkg_dict) File "/usr/lib/ckan/default/src/ckanext-spatial/ckanext/spatial/plugin/__init__.py", line 175, in before_dataset_index pkg_dict = search_backends[search_backend]().index_dataset(pkg_dict) File "/usr/lib/ckan/default/src/ckanext-spatial/ckanext/spatial/search/__init__.py", line 178, in index_dataset shape = self.shape_from_geometry(geometry) File "/usr/lib/ckan/default/src/ckanext-spatial/ckanext/spatial/search/__init__.py", line 37, in shape_from_geometry shape = shapely.geometry.shape(geometry) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/geo.py", line 102, in shape return Polygon(ob["coordinates"][0], ob["coordinates"][1:]) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/polygon.py", line 230, in __new__ shell = LinearRing(shell) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/polygon.py", line 93, in __new__ coordinates = np.array([_coords(o) for o in coordinates]) TypeError: 'float' object is not iterable Indexing dataset 2/32024-03-13 02:12:15,357 ERROR [ckan.lib.search] Error while indexing dataset d6ea3142-0746-4b43-8fdf-cdd2a75643f7: TypeError("'float' object is not iterable") 2024-03-13 02:12:15,363 ERROR [ckan.lib.search] Traceback (most recent call last): File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/__init__.py", line 253, in rebuild package_index.update_dict( File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/index.py", line 106, in update_dict self.index_package(pkg_dict, defer_commit) File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/index.py", line 290, in index_package pkg_dict = item.before_dataset_index(pkg_dict) File "/usr/lib/ckan/default/src/ckanext-spatial/ckanext/spatial/plugin/__init__.py", line 175, in before_dataset_index pkg_dict = search_backends[search_backend]().index_dataset(pkg_dict) File "/usr/lib/ckan/default/src/ckanext-spatial/ckanext/spatial/search/__init__.py", line 178, in index_dataset shape = self.shape_from_geometry(geometry) File "/usr/lib/ckan/default/src/ckanext-spatial/ckanext/spatial/search/__init__.py", line 37, in shape_from_geometry shape = shapely.geometry.shape(geometry) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/geo.py", line 108, in shape return MultiPolygon([[c[0], c[1:]] for c in ob["coordinates"]]) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/multipolygon.py", line 81, in __new__ p = polygon.Polygon(shell, holes) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/polygon.py", line 230, in __new__ shell = LinearRing(shell) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/polygon.py", line 93, in __new__ coordinates = np.array([_coords(o) for o in coordinates]) TypeError: 'float' object is not iterable Indexing dataset 3/32024-03-13 02:12:15,376 ERROR [ckan.lib.search] Error while indexing dataset de972abb-6e84-4bcb-a4d2-cf23a7e306b9: TypeError("'float' object is not iterable") 2024-03-13 02:12:15,381 ERROR [ckan.lib.search] Traceback (most recent call last): File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/__init__.py", line 253, in rebuild package_index.update_dict( File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/index.py", line 106, in update_dict self.index_package(pkg_dict, defer_commit) File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/index.py", line 290, in index_package pkg_dict = item.before_dataset_index(pkg_dict) File "/usr/lib/ckan/default/src/ckanext-spatial/ckanext/spatial/plugin/__init__.py", line 175, in before_dataset_index pkg_dict = search_backends[search_backend]().index_dataset(pkg_dict) File "/usr/lib/ckan/default/src/ckanext-spatial/ckanext/spatial/search/__init__.py", line 178, in index_dataset shape = self.shape_from_geometry(geometry) File "/usr/lib/ckan/default/src/ckanext-spatial/ckanext/spatial/search/__init__.py", line 37, in shape_from_geometry shape = shapely.geometry.shape(geometry) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/geo.py", line 108, in shape return MultiPolygon([[c[0], c[1:]] for c in ob["coordinates"]]) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/multipolygon.py", line 81, in __new__ p = polygon.Polygon(shell, holes) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/polygon.py", line 230, in __new__ shell = LinearRing(shell) File "/usr/lib/ckan/default/lib/python3.8/site-packages/shapely/geometry/polygon.py", line 93, in __new__ coordinates = np.array([_coords(o) for o in coordinates]) TypeError: 'float' object is not iterable 2024-03-13 02:12:15,382 INFO [ckan.lib.search] Finished rebuilding search index.