-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Tag privacy Violators #3347
base: master
Are you sure you want to change the base?
Tag privacy Violators #3347
Conversation
When doing a search this adds some latency for each new IP address right? I mean, it has to do a WHOIS before rendering the results to the user screen. If so, IMO this feature just be configurable and disabled by default. It already takes quite an amount of time to do the requests to the configured engines, so adding even more latency will degrade the user experience. |
I've implemented some kind of caching so when the user sends a query and sees some results, the privacy violators' URLs would be cached, not adding more latency the next time the user sends another query. |
Adding some caching is minor in a case where the URL is always going to be different, when the user search on SearX it is very unlikely it will always search for the same exact thing. Also, most of the engines give different result URLs when you search for the same keyword. The cache will hardly ever get used. On top of that, due to the cache, this feature may decrease the privacy of the user because literally every search the user is doing is going to be saved in memory. As discussed with @dalf about this feature, the best option for the WHOIS is to instead use a local database like this one: https://lite.ip2location.com/database-asn. But it will require a manual intervention from the user in order to setup this database. But using this database will be so much quicker to access than doing a WHOIS request to an external server. |
I agree with you, caching will harm the user's privacy and i will remove it. |
What does this PR do?
This PR adds the ability to tag privacy violating websites, Fixing Issue #1987. I grabbed the ASN data from here, I also used IPWhois as it was used in here. I used a dictionary as a cache so the IPWhois queries wouldn't slow down the searches drastically.
I also changed the templates to show the tagged websites correctly.
Why is this change important?
This PR is motivated by Issue #1987.
How to test this PR locally?
Related issues
Closes #1987