JSOCClient fetch sometimes returns hmi.sharp_720s_nrt bitmap files in the wrong file name format

[cdidigu674]

Describe the bug

hmi.sharp_720s_nrt files when downloaded via JSOCClient fetch should have the following file name format: hmi.sharp_720s_nrt.{HARPNUM}.{T_REC:A}.{segment}. However I have observed a couple of times when the files were in the wrong file name format: hmi.sharp_720s_nrt.{recnum:%d}.{segment}. I have only observed this with the bitmap files.

Looking at the drms source code briefly shows that if filenamefmt is not passed in to the export function (which JSOCClient request_data does not), then the prime keys for the series should be used to form the filename format. The prime keys for the series are HARPNUM and T_REC. However, we are seeing a few instances where the format is wrong. It seems perhaps none of these prime keys are available to be used to form the file name format in drms client.py function _generate_filenamefmt.

Has this behavior been observed before? Is this a JSOC API issue as the prime keys are retrieved from JSOC?

To Reproduce

No response

Screenshots

No response

System Details

sunpy version 4.0.5

Installation method

No response

[nabobalis]

Can please provide a query example I can run to try and work out what is going on?

[cdidigu674]

Here is a cut down from the code we are running. Please replace the notify email with yours. This works 99% of the time, so I don't know if you will be able to recreate:

import astropy.units as u
from sunpy.net import attrs as a
from sunpy.net import jsoc

client = jsoc.JSOCClient()
response = client.search(a.Time("2023.11.02_02:00:00_TAI", "2023.11.02_03:55:03_TAI"),
a.jsoc.Series("hmi.sharp_720s_nrt"),
a.jsoc.Notify("your notify email address"),
a.Sample(1*u.hour),
a.jsoc.Segment("bitmap"))

client.fetch(response, path="./")

[nabobalis]

Works 99% of the time?

[cdidigu674]

Yes most of the time it works. Such a simple logic (forming the file name format) should work 100% of the time.

[nabobalis]

Hmm, running this a few times, I see all of the files following the format as required.

If its failing, its possible it's just due to the return from the JSOC from time to time but that would be my speculation more than anything else.

I'll continue to see if I can get it to fail.

[cdidigu674]

Hello @nabobalis any progress? Did you see anything in the code that might point to why the unsupported "47" is showing up in the filenamefmt of the failing request? And whether or not the correct prime keys were returned from jsoc_info?

[nabobalis]

Hi @cdidigu674.

Unfortunately not, I was not able to replicate it and I spoke to the JSOC, so the problem is definitely on our side, sunpy/jsocclient/drms provided an query with incorrect filename template.

I think this should only happen if there was a timeout from that jsoc_info but I think that would raise another error. So I am still looking into it but right now I am finding it tricky to puzzle out.

[cdidigu674]

Ok sounds good. Thanks for the update.

[nabobalis]

I still have been unable to work to the issue here. Having spoke to the JSOC, it would suggest there might be something wrong in the sunpy/DRMS library but still unable to confirm where.