Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement collapse_particles=False option for InferenceConfig #5

Open
alex-lew opened this issue Mar 4, 2021 · 0 comments
Open

Implement collapse_particles=False option for InferenceConfig #5

alex-lew opened this issue Mar 4, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@alex-lew
Copy link
Contributor

alex-lew commented Mar 4, 2021

In the terminology of the paper, the steps of PClean's SMC initialization algorithm each correspond to a subproblem. Several subproblems together form an increment of the database. Currently, the number of particles is shrunk to 1 after every increment, before cloning back to the original number of particles for the following increment. This enables all particles to share memory for the "distant past" (on which they are forced to agree), and only store "diffs" to that history recording the values on which they may disagree. Without this, each particle would need to store its own copy of the latent database, which is not practical for very large datasets. However, this aspect of the algorithm should probably be configurable, via a collapse_particles=False option in InferenceConfig, which would have each particle store the entire latent database in memory. Future work could explore more memory-efficient representations (I'm sure there are interesting data structures for this sort of thing).

@alex-lew alex-lew added the enhancement New feature or request label Mar 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant