Formatter: np.array formatting
#8452
Replies: 8 comments 18 replies
-
|
Personally I would favour computing the minimum width for each column, as our arrays for Biopython are mainly in test cases, which rarely change. |
Beta Was this translation helpful? Give feedback.
-
|
Also I like the suggested layout: np.array([
# rows here
])i.e. Putting the ( and [ on the same line not to waste vertical space. |
Beta Was this translation helpful? Give feedback.
-
|
What if there are other arguments to the constructor? Like np.array([
[0, 0, 0, 0],
[0, 0, 0, 0]
], dtype=np.uint8) |
Beta Was this translation helpful? Give feedback.
-
|
I suggest to format it the same as other function calls where one argument "expands" over multiple lines: np.array(
[
[0, 0, 0, 0],
[0, 0, 0, 0],
],
dtype=np.uint8,
) |
Beta Was this translation helpful? Give feedback.
-
|
That'd be OK in my personal opinion, however I would prefer less vertical space and less indentation (more like the numpy repr). In my own usage an explicit |
Beta Was this translation helpful? Give feedback.
-
|
Answering the Open Questions:
|
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Oh, OK. I can see some reasons for decoupling the array layout from the global line length limit. But I would set the limit in terms of number of characters required (or line width), not number of elements (column count). This is partly because integers vs floats can make a big difference, and also arrays could be defined at several levels of indentation. I think lists of lists vs lists of lists of numbers is a marginal call (not something I think affects my code or projects much). I have no strong view here. |
Beta Was this translation helpful? Give feedback.
-
The idea isn't to introduce a new option but instead ignore the line-length entirely. It's the only way how we can guarantee that each row is on its own line. |
Beta Was this translation helpful? Give feedback.
-
|
OK - no limits so each row is one line does make sense as a default. Very large arrays/rows are going to be problematic anyway so the author may opt out of automatic formatting here. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah. I don't have experience on how common the very large rows is or if it would still be preferred to have them as single row than the one line per row formatting (which gets very long as well). That's where Users can enforce the one-line per cell layout by adding a trailing semicolon to a row: np.array([
[1, 2,]
])
# becomes
np.array([
[
1,
2,
]
]) |
Beta Was this translation helpful? Give feedback.
-
|
Should this also handle arrays >2D? I guess the innermost list can be formatted as rows? Note that the numpy array representation does something similar by default (each row on a new line, columns right justified & all padded to the width of the maximum value) The relevant function I think is https://numpy.org/doc/stable/reference/generated/numpy.array_repr.html, which has a lot of options and handling for different data types (see https://numpy.org/doc/stable/reference/generated/numpy.set_printoptions.html#numpy.set_printoptions) |
Beta Was this translation helpful? Give feedback.
-
|
Maybe the style should be based based on the numpy repr (which is more compact than the examples given earlier)? |
Beta Was this translation helpful? Give feedback.
-
|
Interesting. The challenge I see with formatting is that we can't pass options. So it would be necessary to pick a good default. |
Beta Was this translation helpful? Give feedback.
-
The next line matches the indentation of the previous. Compare This is sort of against the black style isn't it? It seems semantically like do_thing(first_arg,
second_arg)You could ignore "coalescing" the brackets and just go for a style like
I come across them quite a bit I guess? The nd structure is definitely a core concept in numpy. We could check the source of some popular projects to see how often they are written out - eg. scipy/numpy |
Beta Was this translation helpful? Give feedback.
-
|
Numpy isn't the only array library out there, although it is the most widespread one. For example, I work on Scipp which adds a few extra properties to arrays. We face similar issues with formatting 2d arrays. So I was wondering if it is possible to find a common solution. Two solution that come to my mind are
The latter might provide a better out-of-the-box experience. Conceptually, I don't see why a numpy array constructed from a list would be treated differently from a plain list. (Of course there may be technical reason for why detecting lists of lists of numbers might not be feasible.) For context, in Scipp, creating an array might look like this: sc.array(
dims=['x', 'y'],
values=[[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5]],
unit='m',
)and potentially even with two lists: sc.array(
dims=['x', 'y'],
values=[[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5]],
variances=[[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5]],
unit='m',
)In this case, all arguments are required to be keywords. And there are more functions that take lists like this as arguments. |
Beta Was this translation helpful? Give feedback.
-
From a user perspective this approach seems to me as the most consistent one with the least amount of surprising behavior. I'd imagine an option like |
Beta Was this translation helpful? Give feedback.
-
The main reason is that the proposed design trades keeping the code in the configured line length with keeping rows together. Keeping the code inside the configured line length (where possible) is a fundamental objective of the formatter. Removing this entirely for all multidimensional, number only-lists, may be undesired for many. Altough I don't have any data on this. The other challenge I see is that having multiple opening/closing parentheses on the same line is unprecedented until now and deviates from other formatting and would also be different from how multidimensional non-number lists would be formatted. |
Beta Was this translation helpful? Give feedback.
-
|
@MichaReiser I think his point is that whatever you do for numpy arrays you should also do for plain lists. Has there been any justification for deviation? As for line length, you can still have a grid formatting that that obeys the maximum line length: I don't think you should remove the line length limit even for the grid formatting that you proposed. (Or at least that's a separate proposal.)
Yes, it's a different style. The multiple open-close parentheses are simply an example of vertical formatting. Currently Ruff only supports hanging formatting. And yes, I think if you're going to allow vertical formatting in some circumstances, it should be uniformly adopted rather than special-cased. For example, vertical is superior to hanging (in my opinion) when it uses fewer lines, which happens when it induces no additional line-wraps. |
Beta Was this translation helpful? Give feedback.
-
Yes, this is what I meant. And I agree that removing the line length limit for grid formatting may not be ideal in general. On the other hand, 2d arrays are usually easier to read with vertical scrolling than with line breaks. But I don't have enough experience to judge how large those grids typically are and whether 80 or so columns are enough. Concerning multiple closing parentheses, this is what numpy itself does: >>> np.ones((4, 4))
array([[1., 1., 1., 1.],
[1., 1., 1., 1.],
[1., 1., 1., 1.],
[1., 1., 1., 1.]]) |
Beta Was this translation helpful? Give feedback.
-
|
While
# 2D with trailing comma
np.array(
[[1, 0],
[0, 1]],
)
# 2D without trailing comma
np.array([[1, 0],
[0, 1]])
# With dtype and trailing comma
np.array(
[[1, 0],
[0, 1]],
dtype='int64',
)
# With dtype and without trailing comma
np.array([[1, 0],
[0, 1]], dtype='int64')This already gets us sensible behavior of array-like literals and keyword arguments in the same constructor. But we don't need to special case sc.array(
dims=['x', 'y'],
values=[[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5]],
variances=[[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5]],
unit='m',
)I know they're out-of-scope, but expand here to see the extension to 3- and 4D arrays:# 3D
array(
[[[1., 0.],
[0., 1.]],
[[0., 1.],
[1., 0.]]],
)
# np.arange(48).reshape((2, 3, 2, 4))
array(
[[[[ 0, 1, 2, 3],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[12, 13, 14, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23]]],
[[[24, 25, 26, 27],
[28, 29, 30, 31]],
[[32, 33, 34, 35],
[36, 37, 38, 39]],
[[40, 41, 42, 43],
[44, 45, 46, 47]]]],
)Applying the rule to the arguments of a list literal gives us 3D array-likes almost by accident: numbers = [
[[1, 2, 3, 4],
[5, 6, 7, 8]],
[[ 9, 10, 11, 12],
[13, 14, 15, 16]],
]And if we use parentheses to indicate that the top-level list is to be treated as a single ND array instead of 2 2D arrays: numbers = (
[[[ 1, 2, 3, 4],
[ 5, 6, 7, 8]],
[[ 9, 10, 11, 12],
[13, 14, 15, 16]]]
)It's getting even more out of the scope of |
Beta Was this translation helpful? Give feedback.
-
|
@effigies I like your suggestion too. This is just vertical formatting, which I proposed here: #8197 Honestly, I don't think this should be special-cased for arrays, but rather a better approach to formatting should be considered that takes vertical space into account. |
Beta Was this translation helpful? Give feedback.
-
|
I don't think it's incompatible with what you proposed, but it's not quite the same, I don't think. And I do think that it is worth special-casing for nested lists, as this is an improvement in the current style as well as under your vertical formatting proposal. If your proposal is adopted and allows this special case to be removed, that would be fine, too. |
Beta Was this translation helpful? Give feedback.
-
Right, but if it's an improvement for lists, it's an improvement in other places like nested generator expressions or parameter specifications. We should be pushign for the general solution IMO. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you all for the excellent feedback. I don't know when I'll get time to look into this closer but I certainly have a better understanding for the problem than before and it might be worth trying to find a formatting that works for all multi dimensional arrays. Altough this may require more time to stabilize, which could justify special casing for scientific usages for now to remove the most common need for suppression comments in the short term. |
Beta Was this translation helpful? Give feedback.
-
|
The
I wonder if the same logic can be used by ruff to format list-of-lists. |
Beta Was this translation helpful? Give feedback.
-
Broader adoption of ruff's formatter in the scientific community is the
np.array(see this thread in the Beta Feedback discussion). The solution used by the community today is to suppress the formatting withfmt:off/fmt:onorfmt:skip. However, Ruff's lack of support for expression-level suppressions results in unnecessarily large suppressed ranges (see this comment in the Suppression Comments discussion).The thread discusses two options:
np.arraynp.array, but without an explicit proposal.I prefer not to disable formatting because it means the formatter fails its main goal --- guaranteed consistent code formatting. It also takes the pressure from us to come up with a style that works well for the majority of cases, leaving the ecosystem in a worse state overall.
I want to use this issue to discuss a concrete style for improving
np.arrayformatting. I have never usednp.array, so I'll make incorrect assumptions. This is why I would love to hear feedback on the proposal from people who have usednp.array. Please also correct me if the goal needs refinement.Goal: Improve formatting of the object passed to
np.arraycalls if it is a two dimensional array.Non-Goal: Improve formatting for one or three and more dimensional arrays. One-dimensional list formatting may need an overhaul overall, but this is not specific to
np.arrayProposal
### New Style Applicability
When should the new style apply (rather than using the generic list formatting)
CallExpressionwhere the function is anAttributeExpr(name=np, attribute=array)skip-magic-trailing-comma=false: No row has a trailing comma (indicating that the values should be split on their ownline)
skip-magic-trailing-comma=true: Ignore the style regardless whether a trailing comma is present or notIt is okay for:
Open Questions
np?Proposed style
Example:
Note: The example assumes the preview "hug list preview style".
Alternatives
Instead of padding all cells to the same width, compute the padding for each column.
Considerations
Beta Was this translation helpful? Give feedback.
All reactions