F401 - update documentation and deprecate `ignore_init_module_imports` #11436

plredmond · 2024-05-15T18:55:43Z

Summary

Update documentation for F401 following recent PRs
- F401 - Distinguish between imports we wish to remove and those we wish to make explicit-exports #11168
- F401 - Recommend adding unused import bindings to __all__ #11314
Deprecate ignore_init_module_imports
- Add a deprecation pragma to the option and a "warn user once" message when the option is used.
Restore the old behavior for stable (non-preview) mode:
- When ignore_init_module_imports is set to true (default) there are no __init_.py fixes (but we get nice fix titles!).
- When ignore_init_module_imports is set to false there are unsafe __init__.py fixes to remove unused imports.
- When preview mode is enabled, it overrides ignore_init_module_imports.
Fixed a bug in fix titles where import foo as bar would recommend reexporting bar as bar. It now says to reexport foo as foo. (In this case we don't issue a fix, fwiw; it was just a fix title bug.)

Test plan

Added new fixture tests that reuse the existing fixtures for __init__.py files. Each of the three situations listed above has fixture tests. The F401 "stable" tests cover:

When ignore_init_module_imports is set to true (default) there are no __init_.py fixes (but we get nice fix titles!).

The F401 "deprecated option" tests cover:

When ignore_init_module_imports is set to false there are unsafe __init__.py fixes to remove unused imports.

These complement existing "preview" tests that show the new behavior which recommends fixes in __init__.py according to whether the import is 1st party and other circumstances (for more on that behavior see: #11314).

codspeed-hq · 2024-05-15T19:00:45Z

CodSpeed Performance Report

Merging #11436 will not alter performance

_{Comparing ruff.all (6028415) with main (403f0dc)}

Summary

✅ 30 untouched benchmarks

github-actions · 2024-05-15T19:09:30Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

crates/ruff/tests/snapshots/integration_test__rule_f401.snap

crates/ruff_linter/src/rules/pyflakes/rules/unused_import.rs

plredmond · 2024-05-15T20:36:58Z

@zanieb thoughts on how to gracefully deprecate lint.ignore_init_module_imports? I can just grep for it and remove it, but that seems like it would break people who have the option in their config files.

zanieb · 2024-05-18T17:19:26Z

Here's an example of a deprecated setting:

ruff/crates/ruff_workspace/src/options.rs

Lines 428 to 432 in a73b8c8

    
               #[deprecated( 
        
                   since = "0.1.2", 
        
                   note = "The `tab-size` option has been renamed to `indent-width` to emphasize that it configures the indentation used by the formatter as well as the tab width. Please update your configuration to use `indent-width = <value>` instead." 
        
               )] 
        
               pub tab_size: Option<IndentWidth>,

which has a warning at

ruff/crates/ruff_workspace/src/configuration.rs

Lines 423 to 430 in 6ed2482

    
           #[allow(deprecated)] 
        
           let indent_width = { 
        
               if options.tab_size.is_some() { 
        
                   warn_user_once!("The `tab-size` option has been renamed to `indent-width` to emphasize that it configures the indentation used by the formatter as well as the tab width. Please update your configuration to use `indent-width = <value>` instead."); 
        
               } 
        
               options.indent_width.or(options.tab_size) 
        
           };

I believe this one was deprecated in #8082 if that's helpful. I'm actually not entirely sure why we define a message for the deprecation and manually include a separate warning. Perhaps because it was renamed? I'd have to play with this a bit to find the right usage.

I think we should still respect the setting on stable but you can ignore it in preview. In both cases, we should display a warning. In the future, we can make using the setting an error during configuration loading.

… ruff_workspace

…as message to use that instead of the binding name

…mports=false: emit unsafe fixes to remove unused imports even in __init__.py

…emove unused imports even in __init__.py

…files

plredmond · 2024-05-20T23:18:39Z

crates/ruff_linter/src/rules/pyflakes/rules/unused_import.rs

@@ -244,7 +268,8 @@ pub(crate) fn unused_import(checker: &Checker, scope: &Scope, diagnostics: &mut
    }

    let in_init = checker.path().ends_with("__init__.py");
-    let fix_init = checker.settings.preview.is_enabled();
+    let fix_init = !checker.settings.ignore_init_module_imports;
+    let preview_mode = checker.settings.preview.is_enabled();


These three lines look fishy because I'd originally replaced this

let fix_init = !checker.settings.ignore_init_module_imports;

with

let fix_init = checker.settings.preview.is_enabled();

which was perhaps lazy. In this PR I'm restoring the old behavior, and so gave checker.settings.preview.is_enabled(); its own name.

plredmond · 2024-05-20T23:22:15Z

crates/ruff_linter/src/rules/pyflakes/rules/unused_import.rs

@@ -24,6 +24,7 @@ enum UnusedImportContext {
    Init {
        first_party: bool,
        dunder_all_count: usize,
+        ignore_init_module_imports: bool,


This field is required to make fix titles follow the "old" behavior in __init__.py files. When we finally remove the option ignore_init_module_imports then this field ignore_init_module_imports can be deleted too.

plredmond · 2024-05-20T23:23:20Z

crates/ruff_linter/src/rules/pyflakes/rules/unused_import.rs

            });

        // generate fixes that are shared across bindings in the statement
-        let (fix_remove, fix_reexport) = if (!in_init || fix_init) && !in_except_handler {


This section looks like a large change but isn't. I only added preview_mode to the disjunction. Turn on -w to see the diff w/o whitespace changes.

plredmond · 2024-05-20T23:25:58Z

...lakes/snapshots/ruff_linter__rules__pyflakes__tests__preview__F401_F401_24____init__.py.snap

@@ -39,4 +39,4 @@ __init__.py:36:26: F401 `.renamed` imported but unused; consider removing, addin
 36 | from . import renamed as bees  # F401: no fix
   |                          ^^^^ F401
   |
-   = help: Use an explicit re-export: `bees as bees`
+   = help: Use an explicit re-export: `renamed as renamed`


This is the only change in to an existing snapshot in the whole diff. It was a bug where the fix title contained the binding, instead of the module.

crates/ruff/tests/snapshots/integration_test__rule_f401.snap

crates/ruff_workspace/src/configuration.rs

…e per zanie

… rule per zanie

charliermarsh · 2024-05-21T11:54:49Z

Deferring to @zanieb, feel free to merge when approved!

Hnasar · 2024-05-21T13:56:13Z

crates/ruff/tests/snapshots/integration_test__rule_f401.snap

+Fixes to remove unused imports are safe, except in `__init__.py` files.
+
+Applying fixes to `__init__.py` files is currently in preview. The fix offered depends on the
+type of the unused import. Ruff will suggest a safe fix to export first-party imports with
+either a redundant alias or, if already present in the file, an `__all__` entry. If multiple
+`__all__` declarations are present, Ruff will not offer a fix. Ruff will suggest an unsafe fix
+to remove third-party and standard library imports -- the fix is unsafe because the module's
+interface changes.


I think it's good that ruff has different handling of first vs third party imports, but changing behavior based on the name of the module (containing the import) I find unintuitive and I argue is unpythonic.

By that, I mean: if I have mod/__init__.py, is the fix any more or less safe than if I instead named it mod.py? No — the __init__.py is just an implementation detail and in either case, the module's interface changes with any import changes.

It's not a universally-accepted convention that __init__.py should store your entire public API.
For example, it's hinders import times for users of a large package, if the entire public API is placed in __init__.py, because then any import of a submodule will trigger an import __init__.py and thus an import of every module, if everything is reexported.

At most, we have the typing docs (Typing documentation: interface conventions) because there's nothing I can find on python.org that discusses this. But given the typing docs guidance, I would find it slightly easier to teach that:

third party imports: always safe to remove

first party imports: unsafe to remove if contained within the same package (similar to relative import convention from typing spec). Safe otherwise.

Also for mono-repos, there may be dozens or hundreds of first party top-level packages. Therefore this rule should base its decisions on whether an import is intra-package (e.g. a relative import) rather than first vs third party packages.

Thanks for your feedback @Hnasar — would you mind opening a new issue to discuss this preview behavior? I don't think this pull request is a great place for it since it's just updating the docs.

plredmond added 2 commits May 15, 2024 11:40

docs for updated F401 behavior #11168,#11314

e78fee6

cargo insta review -- accept doc changes

c9ac09b

zanieb reviewed May 15, 2024

View reviewed changes

crates/ruff/tests/snapshots/integration_test__rule_f401.snap Outdated Show resolved Hide resolved

zanieb reviewed May 15, 2024

View reviewed changes

crates/ruff/tests/snapshots/integration_test__rule_f401.snap Outdated Show resolved Hide resolved

zanieb reviewed May 15, 2024

View reviewed changes

crates/ruff/tests/snapshots/integration_test__rule_f401.snap Outdated Show resolved Hide resolved

zanieb reviewed May 15, 2024

View reviewed changes

crates/ruff_linter/src/rules/pyflakes/rules/unused_import.rs Show resolved Hide resolved

plredmond added 2 commits May 15, 2024 13:29

wording improvements in after zanie's comments

fb1ac97

cargo insta review -- accept documentation change

7706561

plredmond added 12 commits May 20, 2024 12:21

add deprecation warning for option lint.ignore_init_module_imports in…

36b4331

… ruff_workspace

add module member name field to UnusedImport violation; redundant-ali…

63a9676

…as message to use that instead of the binding name

cargo insta review -- accept change in diff for incorrect fix title

f656ca4

restore deprecated behavior for f401 when linter.ignore_init_module_i…

1bfa7b1

…mports=false: emit unsafe fixes to remove unused imports even in __init__.py

tests for f401 in stable w/o deprecated option

4f0b79e

tests for f401 in stable w/ deprecated option: emit unsafe fixes to r…

3202de8

…emove unused imports even in __init__.py

add deprecation message for ignore-init-module-imports in settings …

97d1526

…files

cargo insta review -- accept changed documentation

8d3ffbd

cargo dev generate-all

249d848

tweak boolean condition

b6871c9

correct documentation formatting

715da2e

cargo insta review -- accept documentation change

21fe25e

plredmond commented May 20, 2024

View reviewed changes

plredmond marked this pull request as ready for review May 20, 2024 23:29

plredmond requested a review from charliermarsh May 20, 2024 23:32

zanieb reviewed May 20, 2024

View reviewed changes

crates/ruff/tests/snapshots/integration_test__rule_f401.snap Outdated Show resolved Hide resolved

zanieb reviewed May 21, 2024

View reviewed changes

crates/ruff_workspace/src/configuration.rs Outdated Show resolved Hide resolved

zanieb reviewed May 21, 2024

View reviewed changes

crates/ruff_workspace/src/configuration.rs Outdated Show resolved Hide resolved

zanieb approved these changes May 21, 2024

View reviewed changes

plredmond added 4 commits May 20, 2024 20:50

emit the deprecation warning if the option is set, regardless of valu…

26f80c1

…e per zanie

improve wording of deprecation warning by using zanie's suggestion

2be489f

list only the option, not information about it, in the docstring of a…

943ee25

… rule per zanie

cargo insta review -- accept change in documentation

6028415

Hnasar reviewed May 21, 2024

View reviewed changes

plredmond merged commit 7225732 into main May 21, 2024
19 checks passed

plredmond deleted the ruff.__all__ branch May 21, 2024 16:23

BrewTestBot mentioned this pull request May 23, 2024

ruff 0.4.5 Homebrew/homebrew-core#172580

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

F401 - update documentation and deprecate `ignore_init_module_imports` #11436

F401 - update documentation and deprecate `ignore_init_module_imports` #11436

plredmond commented May 15, 2024 •

edited

codspeed-hq bot commented May 15, 2024 •

edited

github-actions bot commented May 15, 2024 •

edited

plredmond commented May 15, 2024

zanieb commented May 18, 2024 •

edited

plredmond May 20, 2024 •

edited

plredmond May 20, 2024 •

edited

plredmond May 20, 2024 •

edited

plredmond May 20, 2024

charliermarsh commented May 21, 2024

Hnasar May 21, 2024

Hnasar May 21, 2024

zanieb May 21, 2024

F401 - update documentation and deprecate ignore_init_module_imports #11436

F401 - update documentation and deprecate ignore_init_module_imports #11436

Conversation

plredmond commented May 15, 2024 • edited

Summary

Test plan

codspeed-hq bot commented May 15, 2024 • edited

CodSpeed Performance Report

Merging #11436 will not alter performance

Summary

github-actions bot commented May 15, 2024 • edited

ruff-ecosystem results

Linter (stable)

Linter (preview)

plredmond commented May 15, 2024

zanieb commented May 18, 2024 • edited

plredmond May 20, 2024 • edited

Choose a reason for hiding this comment

plredmond May 20, 2024 • edited

Choose a reason for hiding this comment

plredmond May 20, 2024 • edited

Choose a reason for hiding this comment

plredmond May 20, 2024

Choose a reason for hiding this comment

charliermarsh commented May 21, 2024

Hnasar May 21, 2024

Choose a reason for hiding this comment

Hnasar May 21, 2024

Choose a reason for hiding this comment

zanieb May 21, 2024

Choose a reason for hiding this comment

F401 - update documentation and deprecate `ignore_init_module_imports` #11436

F401 - update documentation and deprecate `ignore_init_module_imports` #11436

plredmond commented May 15, 2024 •

edited

codspeed-hq bot commented May 15, 2024 •

edited

github-actions bot commented May 15, 2024 •

edited

`ruff-ecosystem` results

zanieb commented May 18, 2024 •

edited

plredmond May 20, 2024 •

edited

plredmond May 20, 2024 •

edited

plredmond May 20, 2024 •

edited