You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I didn't manage to reproduce this using contrived cubes, so I can only currently reproduce it from the source data, at work. Simply load the cubes and the problem occurs. I've written a script which demonstrates the problem and will paste that, and the output, below. I also have a workaround at work. I'm not yet sure if I can share it.
Expected behaviour
The data are global lat-lon octants. I believe they should be loaded into sixteen cubes. Eight of them should share a single time and forecast period, with fourteen pressure levels. The other eight should share a different time and forecast period, and have 11 pressure levels.
Actual behaviour
Instead, the data is loading into eight cubes with the third dimension having 25 points - a mashup of the pressure and time coords.
One impact of this: I'm extracting each time point and saving as netcdf. Without the workaround, it produces a file without proper knowledge of the pressure dimension. It's just called "dim1", it's in a seemingly random order, and the points are just indices 0-13. Another impact is that the coord points are a bit of a mess, as you can see in the output below.
Workaround
I've worked around this by using load_raw, and writing a function which groups by time, merging each group separately to force the pressure levels to be merged into a sensible dimension first. The workaround is specific to the data, not a generic fix.
Environment
OS & Version: RHEL 7.9
Iris Version: 3.7
Additional context
code to highlight the problem
#!/usr/bin/env pythonfrompathlibimportPathimportsyssys.path.append('HIDING_PATHS_HERE')
importirisfromiris.cubeimportCubeList# workaround is in herefromgrib_concatimportload_cubes, load_cubes_controlledif__name__=='__main__':
fpaths= [
Path('HIDING_PATHS_HERE/file1'),
Path('HIDING_PATHS_HERE/file2'),
]
# normal iris loadingiris_load_cubes=load_cubes(fpaths)
iris_156_cubes=CubeList([cforciniris_load_cubesif'UNKNOWN LOCAL PARAM 156'inc.name()])
# workaround with controlled mergingcontrolled_merge_cubes=load_cubes_controlled(fpaths)
controlled_156_cubes=CubeList([cforcincontrolled_merge_cubesif'UNKNOWN LOCAL PARAM 156'inc.name()])
print('--- iris load ---\n')
print(f'{len(iris_156_cubes)} cubes\n')
forciniris_156_cubes:
print(repr(c))
print('\ncube 0:\n', iris_156_cubes[0], '\n')
print('forecast_period', iris_156_cubes[0].coord('forecast_period').points)
print('pressure', iris_156_cubes[0].coord('pressure').points)
print('time', iris_156_cubes[0].coord('time').points, '\n')
print('one of the cubes after extracting fp=0:')
iris_fp0=iris_156_cubes.extract(iris.Constraint(forecast_period=0))[0]
print(repr(iris_fp0))
print('forecast_period', iris_fp0.coord('forecast_period').points)
print('pressure', iris_fp0.coord('pressure').points)
print('time', iris_fp0.coord('time').points, '\n')
iris.save(iris_fp0, 'from_iris_load.nc')
print('--- controlled merge ---\n')
print(f'{len(controlled_156_cubes)} cubes\n')
forcincontrolled_156_cubes:
print(repr(c))
print('\ncube 0:\n', controlled_156_cubes[0], '\n')
print('forecast_period', controlled_156_cubes[0].coord('forecast_period').points)
print('pressure', controlled_156_cubes[0].coord('pressure').points)
print('time', controlled_156_cubes[0].coord('time').points, '\n')
print('one of the cubes after extracting fp=0:')
controlled_fp0=controlled_156_cubes.extract(iris.Constraint(forecast_period=0))[0]
print(repr(controlled_fp0))
print('forecast_period', controlled_fp0.coord('forecast_period').points)
print('pressure', controlled_fp0.coord('pressure').points)
print('time', controlled_fp0.coord('time').points, '\n')
iris.save(controlled_fp0, 'from_controlled_merge.nc')
In the output below, you can first see the data loaded with iris.load, then using iris.load_raw and "controlled" merging. In both cases one cube is printed, and then cubes are extracted at forecast_period=0 with one of those printed.
output
--- iris load ---
8 cubes
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (-- : 25; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (-- : 25; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (-- : 25; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (-- : 25; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (-- : 25; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (-- : 25; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (-- : 25; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (-- : 25; latitude: 61; longitude: 60)>
cube 0:
UNKNOWN LOCAL PARAM 156.128 / (unknown) (-- : 25; latitude: 61; longitude: 60)
Dimension coordinates:
latitude - x -
longitude - - x
Auxiliary coordinates:
forecast_period x - -
pressure x - -
time x - -
Scalar coordinates:
originating_centre European Centre for Medium Range Weather Forecasts
Attributes:
generatingProcessIdentifier 154
indicatorOfParameter 156
indicatorOfTypeOfLevel 'pl'
originating_centre_id 'ecmf'
forecast_period [ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 24 24 24 24 24 24 24 24 24 24
24]
pressure [ 300 250 400 1000 700 925 200 850 500 100 150 50 70 30
700 925 300 200 850 1000 100 400 500 250 150]
time [474240. 474240. 474240. 474240. 474240. 474240. 474240. 474240. 474240.
474240. 474240. 474240. 474240. 474240. 474264. 474264. 474264. 474264.
474264. 474264. 474264. 474264. 474264. 474264. 474264.]
one of the cubes after extracting fp=0:
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (-- : 14; latitude: 61; longitude: 60)>
forecast_period [0 0 0 0 0 0 0 0 0 0 0 0 0 0]
pressure [ 300 250 400 1000 700 925 200 850 500 100 150 50 70 30]
time [474240. 474240. 474240. 474240. 474240. 474240. 474240. 474240. 474240.
474240. 474240. 474240. 474240. 474240.]
--- controlled merge ---
16 cubes
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 14; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 14; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 14; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 14; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 14; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 14; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 14; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 14; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 11; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 11; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 11; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 11; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 11; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 11; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 11; latitude: 61; longitude: 60)>
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 11; latitude: 61; longitude: 60)>
cube 0:
UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 14; latitude: 61; longitude: 60)
Dimension coordinates:
pressure x - -
latitude - x -
longitude - - x
Scalar coordinates:
forecast_period 0 hours
originating_centre European Centre for Medium Range Weather Forecasts
time 2024-02-07 00:00:00
Attributes:
generatingProcessIdentifier 154
indicatorOfParameter 156
indicatorOfTypeOfLevel 'pl'
originating_centre_id 'ecmf'
forecast_period [0]
pressure [ 30 50 70 100 150 200 250 300 400 500 700 850 925 1000]
time [474240.]
one of the cubes after extracting fp=0:
<iris 'Cube' of UNKNOWN LOCAL PARAM 156.128 / (unknown) (pressure: 14; latitude: 61; longitude: 60)>
forecast_period [0]
pressure [ 30 50 70 100 150 200 250 300 400 500 700 850 925 1000]
time [474240.]
The text was updated successfully, but these errors were encountered:
馃悰 Bug Report
Merge is mashing two dimensions into one.
How To Reproduce
I didn't manage to reproduce this using contrived cubes, so I can only currently reproduce it from the source data, at work. Simply load the cubes and the problem occurs. I've written a script which demonstrates the problem and will paste that, and the output, below. I also have a workaround at work. I'm not yet sure if I can share it.
Expected behaviour
The data are global lat-lon octants. I believe they should be loaded into sixteen cubes. Eight of them should share a single time and forecast period, with fourteen pressure levels. The other eight should share a different time and forecast period, and have 11 pressure levels.
Actual behaviour
Instead, the data is loading into eight cubes with the third dimension having 25 points - a mashup of the pressure and time coords.
One impact of this: I'm extracting each time point and saving as netcdf. Without the workaround, it produces a file without proper knowledge of the pressure dimension. It's just called "dim1", it's in a seemingly random order, and the points are just indices 0-13. Another impact is that the coord points are a bit of a mess, as you can see in the output below.
Workaround
I've worked around this by using
load_raw
, and writing a function which groups by time, merging each group separately to force the pressure levels to be merged into a sensible dimension first. The workaround is specific to the data, not a generic fix.Environment
Additional context
code to highlight the problem
In the output below, you can first see the data loaded with
iris.load
, then usingiris.load_raw
and "controlled" merging. In both cases one cube is printed, and then cubes are extracted at forecast_period=0 with one of those printed.output
The text was updated successfully, but these errors were encountered: