Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory usage #1417

Open
dswd opened this issue Sep 16, 2016 · 2 comments
Open

Memory usage #1417

dswd opened this issue Sep 16, 2016 · 2 comments

Comments

@dswd
Copy link
Contributor

dswd commented Sep 16, 2016

Sometimes ToMaTo backend-core uses huge amounts of memory and continues to grow until it crashes.
I suspect that there is a data structure that (under some circumstances) grows infinitely and something is storing new data in a hot loop.

We should check the code for
a) Potentially hot loops that could generate huge amounts of data in short time
b) Unbounded data structures that can grow infinitely over time

I think we might find something in the error handling code.

@a-gerhard
Copy link
Contributor

the dump limit may have fixed that, lets wait until a new version is rolled out on the server.

@a-gerhard
Copy link
Contributor

We could try to use sys.getsizeof to determine the size of the cache, the dumps library's dumps, and some other stuff. Maybe that could give us some clues...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants