Support for .netrc files for authentication?

[liamtoney]

Problem

EarthScope (formerly IRIS) provides .netrc files for restricted data access. These have the form

machine service.iris.edu login <user> password <password>

where <user> and <password> are the arguments taken by obspy.clients.fdsn.client.Client (hereafter, Client). The .netrc file can be conveniently downloaded from the SAGE Data Services account:

It would be convenient if the Client checked for ~/.netrc in its __init__() method and if an entry was present which matched self.base_url, it populated user and password automatically. This would allow folks to avoid providing user and password as arguments or via the set_credentials() method — therefore, helping us avoid sticking such credentials in the code itself.

Then we could ask EarthScope to update the text here 😉

ObsPy does not currently permit the use of a .netrc or other private file to keep the username and password secret (unless you are using EIDA tokens).

Proposed solution

Here's a proof-of-concept snippet that could be adopted into the __init__() method to accomplish this. It uses the base_url attribute to determine which host to grab credentials for.

import netrc

base_url = 'http://service.iris.edu'  # In reality this is self.base_url, AFTER it's been mapped

try:
    n = netrc.netrc()  # Looks for ~/.netrc by default
    host = base_url.replace('http://', '').replace('https://', '')  # Strip URL
    host_credentials = n.authenticators(host)
    if host_credentials:  # This is None if no match in .netrc file
        user = host_credentials[0]
        password = host_credentials[2]
except FileNotFoundError:  # If no ~/.netrc
    pass  # Just carry on

We'd want to not overwrite the user and password if they're provided to __init__() (i.e., if they're not None) of course.

[megies]

I don't see why we shouldn't add support for that. It might be a little bit more complicated to add though, when you look at where authentication is handled, probably adding to the password manager or something, didn't look into all the details:

obspy/obspy/clients/fdsn/client.py

Lines 341 to 360 in 3ae0030

	def _set_opener(self, user, password):
	# Only add the authentication handler if required.
	handlers = []
	if user is not None and password is not None:
	# Create an OpenerDirector for HTTP Digest Authentication
	password_mgr = urllib_request.HTTPPasswordMgrWithDefaultRealm()
	password_mgr.add_password(None, self.base_url, user, password)
	handlers.append(urllib_request.HTTPDigestAuthHandler(password_mgr))
	if (user is None and password is None) or self._force_redirect is True:
	# Redirect if no credentials are given or the force_redirect
	# flag is True.
	handlers.append(CustomRedirectHandler())
	else:
	handlers.append(NoRedirectionHandler())
	# Don't install globally to not mess with other codes.
	self._url_opener = urllib_request.build_opener(*handlers)
	if self.debug:
	print('Installed new opener with handlers: {!s}'.format(handlers))

Also, we would definitely want to use urllib.parse.urlparse to canonically interpret the base_url (scheme) to compare with netrc.

In [2]: urllib.parse.urlparse('http://service.iris.edu')
Out[2]: ParseResult(scheme='http', netloc='service.iris.edu', path='', params='', query='', fragment='')

[liamtoney]

Also, we would definitely want to use urllib.parse.urlparse to canonically interpret the base_url (scheme) to compare with netrc.

Ah, nice. Then that line in my example is:

host = urlparse(base_url).netloc  # Parse URL

Noted on the possibly more complicated implementation details. I'll take a look at this when I have time...