Engines cannot retrieve results: flickr (unexpected crash), google images (unexpected crash)

[Joukahainen123]

This is fresh install of searx 1.1.0 running on Ubuntu 22.04 (in AWS).

For some reason the Google and Flickr image searches are not working. Seeing following errors in /var/log/uwsgi/app/searx.log

ERROR:searx.search.processor.online:engine flickr : exception : invalid literal for int() with base 10: 'data'

Traceback (most recent call last):
File "/usr/local/searx/searx-src/searx/search/processors/online.py", line 144, in search
search_results = self._search_basic(query, params)
File "/usr/local/searx/searx-src/searx/search/processors/online.py", line 128, in _search_basic
return self.engine.response(response)
File "/usr/local/searx/searx-src/searx/engines/google_images.py", line 193, in response
pub_source = extract_text(pub_nodes[1])
IndexError: list index out of range

[Joukahainen123]

Addition. Tried also with Ubuntu 20.04. The same problem (flickr and , google images, unexpected crash).

[Joukahainen123]

Did activate the debug mode. Got following logs out of it.

DEBUG:urllib3.connectionpool:https://www.bing.com:443 "GET /images/search?q=temp&count=28&first=1&tsc=ImageHoverTitle HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://www.google.com:443 "GET /search?q=temp&tbm=isch&lr=en-US&hl=en-US&ie=utf8&oe=utf8&ucbcd=1&num=30 HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://loc.gov:443 "GET /pictures/search/?&sp=1&q=temp&fo=json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://www.deviantart.com:443 "GET /search/deviations?page=1&q=temp HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://api.openverse.engineering:443 "GET /v1/images/?page=1&page_size=20&format=json&q=temp HTTP/1.1" 200 None
DEBUG:searx:google images: ErrorContext('/usr/local/searx/searx-src/searx/engines/google_images.py', 193, 'pub_source = extract_text(pub_nodes[1])', 'IndexError', None, ())
ERROR:searx.search.processor.online:engine google images : exception : list index out of range
Traceback (most recent call last):
File "/usr/local/searx/searx-src/searx/search/processors/online.py", line 144, in search
search_results = self._search_basic(query, params)
File "/usr/local/searx/searx-src/searx/search/processors/online.py", line 128, in _search_basic
return self.engine.response(response)
File "/usr/local/searx/searx-src/searx/engines/google_images.py", line 193, in response
pub_source = extract_text(pub_nodes[1])
IndexError: list index out of range
DEBUG:urllib3.connectionpool:https://www.flickr.com:443 "GET /search?text=temp&page=1 HTTP/1.1" 200 None
DEBUG:searx:flickr: ErrorContext('/usr/local/searx/searx-src/searx/engines/flickr_noapi.py', 79, "photo = model_export['main'][index[0]][int(index[1])][index[2]][index[3]][int(index[4])]", 'ValueError', None, ())
ERROR:searx.search.processor.online:engine flickr : exception : invalid literal for int() with base 10: 'data'
Traceback (most recent call last):
File "/usr/local/searx/searx-src/searx/search/processors/online.py", line 144, in search
search_results = self._search_basic(query, params)
File "/usr/local/searx/searx-src/searx/search/processors/online.py", line 128, in _search_basic
return self.engine.response(response)
File "/usr/local/searx/searx-src/searx/engines/flickr_noapi.py", line 79, in response
photo = model_export['main'][index[0]][int(index[1])][index[2]][index[3]][int(index[4])]
ValueError: invalid literal for int() with base 10: 'data'

[kj-sh604]

Facing the same issue on my instance.

My Instance:

• Long-running install of searx on an Ubuntu 22.04 VPS.
• Server got upgraded from 20.04 LTS around 5 months ago and the initial searx install was 1.0.0.

Relevant Debug Information(?)
I did a search for "EM1 Civic SI"
↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓

> python webapp.py
DEBUG:searx.webapp:http%3A//justsearx.xyz/autocompleter%3Fq%3Dem1%2520civic uses locale `en` from browser
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): suggestqueries.google.com:443
DEBUG:urllib3.connectionpool:https://query.wikidata.org:443 "POST /sparql HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://www.bing.com:443 "GET /search?q=language%3AEN+em1+civic&first=1 HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://www.google.com:443 "GET /search?q=em1+civic&lr=en&hl=en&ie=utf8&oe=utf8&start=0&filter=0&ucbcb=1&async=use_ac%3Atrue%2C_fmt%3Apc HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://search.yahoo.com:443 "GET /search?p=em1+civic&ei=UTF-8&fl=1&vl=lang_en&btf=&fr2=time&age=&b=1&xargs=0 HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://en.wikipedia.org:443 "GET /api/rest_v1/page/summary/Em1%20Civic HTTP/1.1" 301 0
DEBUG:urllib3.connectionpool:https://en.wikipedia.org:443 "GET /api/rest_v1/page/summary/Em1_Civic HTTP/1.1" 404 193
DEBUG:urllib3.connectionpool:https://suggestqueries.google.com:443 "GET /complete/search?client=toolbar&hl=en&q=em1+civic HTTP/1.1" 200 None
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:12] "GET /autocompleter?q=em1%20civic HTTP/1.1" 200 -
DEBUG:urllib3.connectionpool:https://www.reddit.com:443 "GET /search.json?q=em1+civic&limit=25 HTTP/1.1" 200 24592
DEBUG:urllib3.connectionpool:https://search.brave.com:443 "GET /search?q=em1%2Bcivic HTTP/1.1" 200 None
DEBUG:searx.google engine:did not find 'answer'
DEBUG:searx.google engine:add link to results: EM1 Honda Civic Si vs EK9 Honda Civic Type R
DEBUG:searx.google engine:add link to results: Used 2000 Honda Civic Si Coupe 2D - Kelley Blue Book
DEBUG:searx.google engine:add link to results: Honda Civic (sixth generation) - Wikipedia
DEBUG:searx.google engine:add link to results: Honda Civic For Sale - BaT Auctions - Bring a Trailer
DEBUG:searx.google engine:ingoring item from the result_xpath list: missing title
DEBUG:urllib3.connectionpool:https://lite.duckduckgo.com:443 "POST /lite HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): duckduckgo.com:443
DEBUG:urllib3.connectionpool:https://duckduckgo.com:443 "GET /t/sl_l HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://api.qwant.com:443 "GET /v3/search/web?q=em1+civic&count=10&offset=0&locale=en_US HTTP/1.1" 200 None
DEBUG:searx.webapp:http%3A//justsearx.xyz/search uses locale `en` from preferences
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:13] "POST /search HTTP/1.1" 200 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/css/bootstrap.min.css uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:13] "GET /static/themes/oscar/css/bootstrap.min.css HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/css/logicodev-dark.min.css uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:13] "GET /static/themes/oscar/css/logicodev-dark.min.css HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/js/jquery.min.js uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:13] "GET /static/themes/oscar/js/jquery.min.js HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/js/bootstrap.min.js uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:13] "GET /static/themes/oscar/js/bootstrap.min.js HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/js/typeahead.bundle.min.js uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:13] "GET /static/themes/oscar/js/typeahead.bundle.min.js HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/plugins/js/search_on_category_select.js uses locale `en` from browser
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/js/searx.min.js uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:13] "GET /static/themes/oscar/js/searx.min.js HTTP/1.1" 304 -
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:13] "GET /static/plugins/js/search_on_category_select.js HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/img/favicon.png uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:13] "GET /static/themes/oscar/img/favicon.png HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/search uses locale `en` from browser
DEBUG:searx.search:actual_timeout=3.0 (default_timeout=3.0, ?timeout_limit=None, max_request_timeout=None)
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.openverse.engineering:443
DEBUG:searx.google images:query_url --> https://www.google.com/search?q=em1+civic&tbm=isch&lr=en-US&hl=en-US&ie=utf8&oe=utf8&ucbcd=1&num=30
DEBUG:searx.google images:HTTP header Accept-Language --> None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): loc.gov:443
DEBUG:urllib3.connectionpool:https://duckduckgo.com:443 "GET /?q=em1+civic&iar=images&iax=1&ia=images HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://www.bing.com:443 "GET /images/search?q=em1+civic&count=28&first=1&tsc=ImageHoverTitle HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://www.google.com:443 "GET /search?q=em1+civic&tbm=isch&lr=en-US&hl=en-US&ie=utf8&oe=utf8&ucbcd=1&num=30 HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://loc.gov:443 "GET /pictures/search/?&sp=1&q=em1+civic&fo=json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://www.reddit.com:443 "GET /search.json?q=em1+civic&limit=25 HTTP/1.1" 200 24698
DEBUG:urllib3.connectionpool:https://api.openverse.engineering:443 "GET /v1/images/?page=1&page_size=20&format=json&q=em1+civic HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://duckduckgo.com:443 "GET /i.js?q=em1+civic&l=wt-wt&s=0&p=-1&o=json&vqd=3-241385915280594941323400282018008232002-26283875701996683966613348965642615259 HTTP/1.1" 200 None
DEBUG:searx:google images: ErrorContext('/usr/local/searx/searx-src/searx/engines/google_images.py', 193, 'pub_source = extract_text(pub_nodes[1])', 'IndexError', None, ())
ERROR:searx.search.processor.online:engine google images : exception : list index out of range
Traceback (most recent call last):
  File "/usr/local/searx/searx-src/searx/search/processors/online.py", line 144, in search
    search_results = self._search_basic(query, params)
  File "/usr/local/searx/searx-src/searx/search/processors/online.py", line 128, in _search_basic
    return self.engine.response(response)
  File "/usr/local/searx/searx-src/searx/engines/google_images.py", line 193, in response
    pub_source = extract_text(pub_nodes[1])
IndexError: list index out of range
DEBUG:urllib3.connectionpool:https://api.qwant.com:443 "GET /v3/search/images?q=em1+civic&count=50&offset=0&locale=en_US HTTP/1.1" 200 None
DEBUG:searx.webapp:http%3A//justsearx.xyz/search uses locale `en` from preferences
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:15] "POST /search HTTP/1.1" 200 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/css/bootstrap.min.css uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:15] "GET /static/themes/oscar/css/bootstrap.min.css HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/css/logicodev-dark.min.css uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:15] "GET /static/themes/oscar/css/logicodev-dark.min.css HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/js/jquery.min.js uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:15] "GET /static/themes/oscar/js/jquery.min.js HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/js/typeahead.bundle.min.js uses locale `en` from browser
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/js/bootstrap.min.js uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:15] "GET /static/themes/oscar/js/typeahead.bundle.min.js HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/js/searx.min.js uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:15] "GET /static/themes/oscar/js/bootstrap.min.js HTTP/1.1" 304 -
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:15] "GET /static/themes/oscar/js/searx.min.js HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/plugins/js/search_on_category_select.js uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:15] "GET /static/plugins/js/search_on_category_select.js HTTP/1.1" 304 -
DEBUG:searx.webapp:http%3A//justsearx.xyz/static/themes/oscar/img/favicon.png uses locale `en` from browser
INFO:werkzeug:65.26.154.27 - - [26/Nov/2022 14:00:16] "GET /static/themes/oscar/img/favicon.png HTTP/1.1" 304 -
^C⏎                                                                                                           

I currently have flickr and google images disabled on my instance due to this bug. However, google images (goi) was enabled during the debug run.

Please let me know if there's any more information I can share to help debug and/or create a workaround for the issue. Very much willing to do so 😌 Thank you for the amazing project nonetheless.

[elliotcourant]

Experiencing the exact same issue; please let me know if there is any additional details I can provide as well. I'm seeing the exact same logs as the ones above. Running 1.1.0-53-c647b55e via docker.

[unixfox]

Have you tried searxng a fork of searx which is more frequently updated and work fine for all the engines listed here: https://github.com/searxng/searxng?

[elliotcourant]

I have not but I will give that a try!