Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel computation precision #304

Open
tensionhead opened this issue Jul 11, 2022 · 2 comments
Open

Parallel computation precision #304

tensionhead opened this issue Jul 11, 2022 · 2 comments
Labels
Explore Examine novel functionality/proposed changes etc. Does not necessarily involve coding things. Performance Improve the number crunching

Comments

@tensionhead
Copy link
Contributor

At least for the coherence, sequential and parallel computing do not give the same results:

import numpy as np                                                                                         
import syncopy as spy                                                                                      
import syncopy.tests.synth_data as synth_data                                                              
import dask.distributed as dd                                                                              
                                                                                                            
adata = synth_data.white_noise(nTrials=10,                                                                 
                                nSamples=100,                                                               
                                nChannels=2,                                                                
                                samplerate=200)                                                             
                                                                                                            
                                                                                                            
def test_seq():                                                                                            
                                                                                                            
    res = spy.connectivityanalysis(adata)                                                                  
    res2 = spy.connectivityanalysis(adata)                                                                 
    print()                                                                                                
    print("all equal:", np.all(res.data[:] == res2.data[:]))                                               
    print()                                                                                                
                                                                                                            
                                                                                                            
def test_parallel():                                                                                       
                                                                                                            
    client = dd.Client()                                                                                   
    test_seq()                                                                                             
    client.close()

This will give True for test_seq but False for test_parallel! No idea where to start to look even, coherences can have tricky denominators (if power at that freq. is almost 0), but no clue if that's the problem here.

@tensionhead tensionhead added the Bug An error that is serious but does not break (parts of) the package. However, it clearly impedes the label Jul 11, 2022
tensionhead added a commit that referenced this issue Jul 11, 2022
- testing single and chained call plus selections
- roadblock atm: #304
@tensionhead tensionhead added Minor Bug A flaw in the code or documentation that is clearly an error but does neither impede package functio and removed Bug An error that is serious but does not break (parts of) the package. However, it clearly impedes the labels Jul 11, 2022
@tensionhead
Copy link
Contributor Author

So as for these connectivity measures the values are all between [-1, 1], and we have errors of about 1e-8 we can probably ignore this. However I'll keep it open, as we did not find the cause for this behavior.

@dfsp-spirit dfsp-spirit changed the title Paralell computation precision Parallel computation precision Aug 25, 2022
@tensionhead
Copy link
Contributor Author

Turns out, for Granger causality that can have effects of up to 1e-2, especially around the 0-frequency. Still not really problematic, but definitely noticable!

tensionhead added a commit that referenced this issue Dec 14, 2022
@tensionhead tensionhead added the Explore Examine novel functionality/proposed changes etc. Does not necessarily involve coding things. label Dec 14, 2022
@tensionhead tensionhead added Performance Improve the number crunching and removed Minor Bug A flaw in the code or documentation that is clearly an error but does neither impede package functio labels Jan 2, 2023
tensionhead added a commit that referenced this issue Apr 3, 2023
- parallel precision issue again, see #304
tensionhead added a commit that referenced this issue Apr 3, 2023
- parallel precision issue again, see #304
tensionhead added a commit that referenced this issue Apr 5, 2023
- we really need to look into this parallel precision issues #304
tensionhead added a commit that referenced this issue Apr 5, 2023
- we really need to look into this parallel precision issues #304
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Explore Examine novel functionality/proposed changes etc. Does not necessarily involve coding things. Performance Improve the number crunching
Projects
None yet
Development

No branches or pull requests

1 participant