A request is issued every time pyingest/config is loaded

[marblestation]

The config file executes a request to github and downloads the list of UAT terms:

adsabs-pyingest/pyingest/config/config.py

Lines 140 to 144 in 15ae056

	uat_request = requests.get(UAT_URL)
	uat_data = uat_request.json()
	get_uat(uat_request.json(), UAT_ASTRO_URI_DICT)
	UAT_ASTRO_KEYWORDS = list(UAT_ASTRO_URI_DICT.keys())
	UAT_ASTRO_URI_DICT = dict((k.lower(), v) for k, v in list(UAT_ASTRO_URI_DICT.items()))

This makes pyingest depend on github being operative, and it can lead to different behaviors with time given that the downloaded file can change and it is not under our control (we cannot guarantee reproducibility). It would be prefered to have a concrete version of that file included in pyingest and update it whenever is necessary (+ commit to our repo and make a new release).

[seasidesparrow]

@aaccomazzi may want to comment here, but this was by design -- this method makes sure the current version is in use whenever ingest is run. Also ADS has substantial control/knowledge of this file and its format because EH/AA are both among the team of the UAT.

[aaccomazzi]

While I understand the reason behind the current implementation, I am also uncomfortable with the dependency on an external resource.

I suggest we change the behavior to use a cached version of the UAT, and have a small setup script that downloads it on demand if and when a curator chooses to do so. Among other things, I know Katie is working on modifying some of the assets within the Github repo and we wouldn't want that to break our pipeline.