Pydicom v3.0 packaging/testing

[darcymason]

Starting a new issue specific to pydicom project layout / packaging / testing setup for v3.0 (#1232).

This also spawned from some discussion in #1801.

Thoughts / discussion:

• require pydicom-data for test running
• create more optional dependency installs -- e.g. pip install pydicom[dev], or pydicom[test] in addition to [docs]. They could include pydicom-data automatically.
• as an aside to the above, perhaps a pydicom[pixeldata] could include numpy and a few libs like the pylibjpeg libraries to handle most compressed images.
• move tests outside the package. That does seem to be the recommendation now, especially for non-small projects, and came up in discussion around TypeError: dcmread: Expected a file ... but got NoneType test failures on =pydicom-2.3.1 #1800.
• consider "src layout" rather than "flat" layout. I don't love it, but pytest argues for the src layout, and apparently some build tools have talked about promoting that as the default. The src layout may be gaining some momentum. If we wanted to go that way, then switching during a major release (and with change of tests location) would be easier than trying to do it later.
• consider modern build tools like flit, hatch, poetry (I've tried poetry and didn't really 'get' it, but if there are strong arguments for it...)

From comment below:

• Remove Pillow support

The result of all the above layout change would give the following folder structure:

- doc
- src (or pydicom)
- tests
- (additional config files etc.)

Which is a fairly clear, logical structure.

[mrbean-bremen]

as an aside to the above, perhaps a pydicom[pixeldata] could include numpy and a few libs like the pylibjpeg libraries to handle most compressed images.

Yes, that is a good idea, at least the openjpeg and the RLE plugins (libjpeg is GPL, so I wouldn't include it by default), plus the jpegls library (which probably could also be made a pylibjpeg plugin, if I think about it).
On a related note: I think I have mentioned that before - I would like to deprecate support for pillow in favor of pylibjpeg and gdcm, as it had too many problem, and installing pygdcm is now trivial.

move tests outside the package.

Yes, I'm all for it - I think moving them into the package had been a mistake.

consider "src layout" rather than "flat" layout

I have no strong opinion here, but if it is the coming standard... And your layout looks ok.

consider modern build tools like flit, hatch, poetry

I would say only if it brings some advantage. But yes, in the long run this makes sense.

Just my quick 2 euro cents, I may have a closer look some time later.