Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Runtime improvements #321

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Conversation

gemeinl
Copy link
Collaborator

@gemeinl gemeinl commented Aug 11, 2021

Reduced runtime of TUHAbonormal from 2 min on master to 44s.
Reduced runtime of TUH from 58 min on master to 10 min.

master:
image
this PR:
image

@codecov
Copy link

codecov bot commented Aug 11, 2021

Codecov Report

Merging #321 (12792fb) into master (9cc8c68) will increase coverage by 0.06%.
The diff coverage is 98.11%.

@@            Coverage Diff             @@
##           master     #321      +/-   ##
==========================================
+ Coverage   83.03%   83.10%   +0.06%     
==========================================
  Files          55       55              
  Lines        3838     3853      +15     
==========================================
+ Hits         3187     3202      +15     
  Misses        651      651              

@gemeinl
Copy link
Collaborator Author

gemeinl commented Aug 11, 2021

Despite the improvements introduced with this PR, load_concat_dataset is still quite slow.
Thereby, close to 100% of the consumed time falls on mne.io.read_raw_fif, so I guess there is not much we can do.

A way faster solution for saving and loading our datasets would be to use pickle.
I am not sure whether we should include / recommend that though...
image

What do you guys think? Any ideas for further speed ups and improvements?

@agramfort
Copy link
Collaborator

agramfort commented Aug 12, 2021 via email

@bruAristimunha bruAristimunha added this to the 0.7 milestone Sep 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants