Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

accelerate some deep scans and safer filter for "*~" backup files #1356

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

verdy-p
Copy link

@verdy-p verdy-p commented Apr 19, 2022

  • Accelerate deep scans by factorizing multiple regexps for the same cleaner option to reduce the number of passes (especially on large filesystems or with mounted physical hard disks or network-mounted drives).
  • Safer filter for "*" backup files (exclude "hidden" filenames starting by a "."; if needed there should be a separate option), but also adds backup filenames ending with a digit before the "" (e.g. "index.php7~", or "README2~")

* Accelerate deep scans by factorizing regexps to reduce the number of passes for the same cleaner option (especially on large filesystems or with mounted physical hard disks or network-mounted drives).
* Safer filter for "*~" backup files (exclude "hidden" filenames starting by a ".")
@az0
Copy link
Member

az0 commented Apr 28, 2022

The idea is interesting, so thanks for working on it.

Please address the errors. Here is one from Travis

======================================================================
ERROR: test_DeepScan (tests.TestDeepScan.DeepScanTestCase)
Unit test for class DeepScan.  Preview real files.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/bleachbit/bleachbit/tests/TestDeepScan.py", line 81, in test_DeepScan
    for cmd in ds.scan():
  File "/home/travis/build/bleachbit/bleachbit/bleachbit/DeepScan.py", line 105, in scan
    compiled_searches = [CompiledSearch(s) for s in searches]
  File "/home/travis/build/bleachbit/bleachbit/bleachbit/DeepScan.py", line 105, in <listcomp>
    compiled_searches = [CompiledSearch(s) for s in searches]
  File "/home/travis/build/bleachbit/bleachbit/bleachbit/DeepScan.py", line 67, in __init__
    self.regex = re_compile(search.regex)
  File "/home/travis/build/bleachbit/bleachbit/bleachbit/DeepScan.py", line 65, in re_compile
    return re.compile(regex, fs_scan_re_flags) if regex else None
  File "/opt/python/3.5.10/lib/python3.5/re.py", line 224, in compile
    return _compile(pattern, flags)
  File "/opt/python/3.5.10/lib/python3.5/re.py", line 293, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/opt/python/3.5.10/lib/python3.5/sre_compile.py", line 536, in compile
    p = sre_parse.parse(p, flags)
  File "/opt/python/3.5.10/lib/python3.5/sre_parse.py", line 829, in parse
    p = _parse_sub(source, pattern, 0)
  File "/opt/python/3.5.10/lib/python3.5/sre_parse.py", line 437, in _parse_sub
    itemsappend(_parse(source, state, nested + 1))
  File "/opt/python/3.5.10/lib/python3.5/sre_parse.py", line 781, in _parse
    source.tell() - start)
sre_constants.error: missing ), unterminated subpattern at position 0
======================================================================
FAIL: test_deep_scan (tests.TestWorker.WorkerTestCase)
Test for deep scan
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/bleachbit/bleachbit/tests/TestWorker.py", line 313, in test_deep_scan
    while next(worker):
  File "/home/travis/build/bleachbit/bleachbit/bleachbit/Worker.py", line 269, in run
    yield from self.run_deep_scan()
  File "/home/travis/build/bleachbit/bleachbit/bleachbit/Worker.py", line 322, in run_deep_scan
    ds = DeepScan.DeepScan(self.deepscans)
  File "/home/travis/build/bleachbit/bleachbit/tests/TestWorker.py", line 301, in __init__
    s.regex, ['^Thumbs\\.db$', '^Thumbs\\.db:encryptable$'])
AssertionError: '^Thumbs\\.db(:encryptable)?$' not found in ['^Thumbs\\.db$', '^Thumbs\\.db:encryptable$']

Full Travis build log: https://app.travis-ci.com/github/bleachbit/bleachbit/jobs/567515304#L1093

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants