About the issue of recording saving.

[yyyaaaaaaa]

When I try to save preprocessed data after extracting the recordings, my terminal always seems unresponsive, as if the entire code has stopped running. However, when I check the saved folder, it already exists. But when I try to extract using the si.load_extractor() , it throws the following error.
Spikeinterface version is V0.100.6.

Traceback (most recent call last):
File "E:\y\python_files\sort\test.py", line 101, in
recording_rec = si.load_extractor(DATA_DIRECTORY / preprocessed)
File "D:\software\Anaconda3\envs\kilosort4\lib\site-packages\spikeinterface\core\base.py", line 1146, in load_extractor
return BaseExtractor.load(file_or_folder_or_dict, base_folder=base_folder)
File "D:\software\Anaconda3\envs\kilosort4\lib\site-packages\spikeinterface\core\base.py", line 781, in load
raise ValueError(f"This folder is not a cached folder {file_path}")
ValueError: This folder is not a cached folder H:\MEA_DATA_binary\yy\20240130\20240130_19531_D13\240130\19531\Network\000015\binary_for_ks4

Here's the script I'm using.

recording_f = bandpass_filter(recording=recording, freq_min=300, freq_max=6000)
recording_cmr = common_reference(recording=recording_f, operator="median")
recording_sub = recording_cmr

preprocessed = "binary_for_ks4"
job_kwargs = dict(n_jobs=30, chunk_duration='1s', progress_bar=True)
rec_saved = recording_sub.save(folder=DATA_DIRECTORY/preprocessed, overwrite=True, format='binary', **job_kwargs)

[alejoe91]

Hi @yyyaaaaaaa

What spikeinterface version are you using?
How large is your recording? If the save function is not printing anything, it means it didn't run successfully and so it's expected that you are not able to reload the extractor

[yyyaaaaaaa]

I'm using version 0.100.6, and this is information about my preprocessed recording.

CommonReferenceRecording: 1012 channels - 20.0kHz - 1 segments - 6,000,200 samples
300.01s (5.00 minutes) - int16 dtype - 11.31 GiB

[alejoe91]

Can you try with n_jobs=1? Just to see if it runs :)

[yyyaaaaaaa]

After waiting for a while, it started working normally. However, it seems a bit slow. Is this normal?

write_binary_recording with n_jobs = 1 and chunk_size = 20000
write_binary_recording: 26%|##5 | 77/301 [04:00<08:53, 2.38s/it]

[alejoe91]

With 1 job it's supposed to be slow. Can you try to gradually increase it? Does it work with 2?

[yyyaaaaaaa]

It's not working. So far, no relevant information has been printed out.

[zm711]

Just for our provenance this is now the 4th case of n_jobs >1 not working on save binary. 3 times on Windows. #2820 is another example. I don't understand the deeper levels of the chunking to try to troubleshoot this. But I think this has to do with some nitty-gritty environment issues on specific computers. For example for my labmate's computer it works in ipython in the terminal but not in an ide.

[yyyaaaaaaa]

Yes, I'm using a Windows system and running my script through PyCharm. Fortunately, setting n_jobs = 1 allows me to work normally, although it's a bit slower :)

[h-mayorquin]

One last questino that will be useful for us. What format is your original data? That is, what format is your original recording? Also, your chunks are I think too smal for writing.

This is a deep issue but I suggest to try to two things.

1. First, on windows, when you run parallel on windows, your script should be protected see this comment in the previous issue:
   Unexpected need to protect lines inside script for multiprocessing #2122
2. Can you show us your system resources as you run the program with this branch?
   Only assign memmap within boundaries for write_binary #2796

I find the later unlikely because your process sould be killed at some point but maybe is overswapping and that's why it becomes so slow.

[h-mayorquin]

Suggestion:
We should do the CI testing with the spawn method to avoid the bias that has creep because Alessio, me and Sam are linux users.

[zm711]

Also, your chunks are I think too smal for writing.

That's our default so if that is the case then it's really our fault for choosing that :)

@h-mayorquin the one linux person who had this fail had an issue with numpy backend stuff see this comment here

[h-mayorquin]

I am linking the numpy issue since it is broken in the other thread:
numpy/numpy#11734

Unfortunatley, I don't think we can safeguard against bugs at the numpy/Intel compilers level.

[zm711]

But in this case I think we need to make sure that people have the option to use n_jobs=1 at each stage. And @alejoe91 and I had previously talked about adding a troubleshooting in the docs to let people know they could look into this for their own computer if they want to try to use the multiprocessing.

[h-mayorquin]

I was talking about the CI. I think that testing for that specific case is too granular even for my preferences, specially if it just hangs.

I think that n_jobs=1 default is a good idea. You will have to convince @samuelgarcia about it though.

[zm711]

Here's another saving issue on Windows I had forgotten about: #1922.

When I have time I might open a separate global issue so I can try to summarize the state of problems with multiprocessing in the repo.