Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Analyzer set recording #2785

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

alejoe91
Copy link
Member

This is useful for example if the analyzer was set up and ran with a tmp cached recording, but later on one wants to set a lazy recording to perform additional operations and computations

@alejoe91 alejoe91 added enhancement New feature or request core Changes to core module labels Apr 30, 2024
@zm711
Copy link
Collaborator

zm711 commented May 2, 2024

I really like this idea because we often do the analysis and then move everything over to our server for long-term storage which means the recording-analyzer link gets broken. My only concern (and this is now from experience helping others) is that people forget what goes with what. So is there any way we can have a safety check? This is honestly the end-users responsibility not to mess up, but are there any attributes on the analyzer that could say help make sure the recording being temporarily used is reasonable?

@alejoe91
Copy link
Member Author

alejoe91 commented May 2, 2024

I really like this idea because we often do the analysis and then move everything over to our server for long-term storage which means the recording-analyzer link gets broken. My only concern (and this is now from experience helping others) is that people forget what goes with what. So is there any way we can have a safety check? This is honestly the end-users responsibility not to mess up, but are there any attributes on the analyzer that could say help make sure the recording being temporarily used is reasonable?

I think that we could check that the recording has the correct recording attributes, i.e., channel_ids, locations, etc

@alejoe91
Copy link
Member Author

@zm711 can you check this again after the last commit?

Copy link
Collaborator

@zm711 zm711 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple comments and questions. But this looks pretty good to me.

src/spikeinterface/core/recording_tools.py Outdated Show resolved Hide resolved
src/spikeinterface/core/recording_tools.py Outdated Show resolved Hide resolved
recording.get_channel_locations(), self.get_channel_locations()
), "Recording channel locations do not match."
if self._recording is not None:
warnings.warn("SortingAnalyzer recording is already set. This will overwrite the current recording.")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
warnings.warn("SortingAnalyzer recording is already set. This will overwrite the current recording.")
warnings.warn("SortingAnalyzer recording is already set. Current recording is being overwritten.")

I switched from future tense to present tense to try to emphasize it is happening. Future tense gives it a vibe that one could prevent this from happening but by the warning it is too late.

Also maybe do we want to say that if save_as it used this new recording will be written to the metadata so use caution?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually reading both our warnings they sound a little frightening. We are not overwriting the recording, but we are point to a new recording. Maybe we think a bit more how to say this without being scary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

temporarily replaced?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that sounds better. I'm still curious what would happen if I do:

analyzer.set_temp_recording(my_new_recording)
analyzer.save_as('memory')
do stuff
analyzer.save_as('zarr', xx)

Will that then write the temp_recording to the file instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's a really good point. Maybe keeping a flag would be good then

src/spikeinterface/core/recording_tools.py Outdated Show resolved Hide resolved
src/spikeinterface/core/sortinganalyzer.py Outdated Show resolved Hide resolved
src/spikeinterface/core/recording_tools.py Outdated Show resolved Hide resolved
recording2_attributes = deepcopy(recording2_attributes)
if skip_properties:
recording1_attributes.pop("properties")
recording2_attributes.pop("properties")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are upi handling the case skip_properties:=false after this ?
I would remove skip_properties option because we do not use.
No ?

Comment on lines +31 to +33
@pytest.fixture(scope="module")
def get_dataset():
return _get_dataset()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@pytest.fixture(scope="module")
def get_dataset():
return _get_dataset()
@pytest.fixture(scope="module")
def dataset():
return get_dataset()

@@ -15,7 +15,7 @@
import numpy as np


def get_dataset():
def _get_dataset():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def _get_dataset():
def get_dataset():

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This smater no ?
why private ?

return _get_dataset()


def test_SortingAnalyzer_memory(tmp_path, get_dataset):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def test_SortingAnalyzer_memory(tmp_path, get_dataset):
def test_SortingAnalyzer_memory(tmp_path, dataset):

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The get_dataset sounds like a function but here it the variable.

test_SortingAnalyzer_memory(tmp_path)
test_SortingAnalyzer_binary_folder(tmp_path)
test_SortingAnalyzer_zarr(tmp_path)
dataset = _get_dataset()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dataset = _get_dataset()
dataset = get_dataset()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Changes to core module enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants