Issue with build_cutout using alite #286

Rock910 · 2023-04-17T16:08:58Z

Using my config.yaml file, I get an error in the build_cutout, which is related to alite

INFO:root:Preparing cutout with parameters {'module': ['sarah', 'era5'], 'x': slice(-12.0, 45.0, None), 'y': slice(33.0, 65, None), 'dx': 0.2, 'dy': 0.2, 'time': slice('2016-01-01', '2016-02-01', None), 'sarah_interpolate': False, 'sarah_dir': None, 'features': ['influx', 'temperature']}.
INFO:atlite.cutout:Building new cutout cutouts/europe-2013-sarah.nc
INFO:atlite.data:Storing temporary files in /tmp/tmpd4bvedp8
INFO:atlite.data:Calculating and writing with module sarah:
Traceback (most recent call last):
File "/home/ubuntu/pypsa-eur/.snakemake/scripts/tmpw6pc4v_v.build_cutout.py", line 134, in
cutout.prepare(features=features)
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/atlite/data.py", line 102, in wrapper
res = func(*args, **kwargs)
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/atlite/data.py", line 164, in cutout_prepare
ds = get_features(cutout, module, missing_features, tmpdir=tmpdir)
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/atlite/data.py", line 46, in get_features
datasets = compute(datasets)
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/dask/base.py", line 599, in compute
results = schedule(dsk, keys, **kwargs)
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/dask/threaded.py", line 89, in get
results = get_async(
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/dask/local.py", line 511, in get_async
raise_exception(exc, tb)
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/dask/local.py", line 319, in reraise
raise exc
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/dask/local.py", line 224, in execute_task
result = _execute_task(task, data)
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/dask/core.py", line 119, in _execute_task
return func((_execute_task(a, cache) for a in args))
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/dask/utils.py", line 73, in apply
return func(args, kwargs)
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/atlite/datasets/sarah.py", line 198, in get_data
files = get_filenames(sarah_dir, coords)
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/atlite/datasets/sarah.py", line 73, in get_filenames
dict(sis=_filenames_starting_with("SIS"), sid=_filenames_starting_with("SID")),
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/site-packages/atlite/datasets/sarah.py", line 62, in _filenames_starting_with
pattern = os.path.join(sarah_dir, "", f"{name}.nc")
File "/home/ubuntu/anaconda3/envs/pypsa-eur/lib/python3.10/posixpath.py", line 76, in join
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType
[Mon Apr 17 15:44:37 2023]
Error in rule build_cutout:
jobid: 14
input: resources/regions_onshore.geojson, resources/regions_offshore.geojson
output: cutouts/europe-2013-sarah.nc
log: logs/build_cutout/europe-2013-sarah.log (check log file(s) for error details)


# SPDX-FileCopyrightText: : 2017-2023 The PyPSA-Eur Authors
#
# SPDX-License-Identifier: CC0-1.0

version: 0.7.0
tutorial: false

logging:
  level: INFO
  format: '%(levelname)s:%(name)s:%(message)s'

run:
  name: "" # use this to keep track of runs with different settings
  shared_cutouts: false # set to true to share the default cutout(s) across runs


scenario:
  simpl: ['']
  ll: ['copt']
  clusters: [37]
  #clusters: [37, 128, 256, 512, 1024]
  opts: [Co2L-1H]
  #opts: [Co2L-3H]

countries: ['AL', 'AT', 'BA', 'BE', 'BG', 'CH', 'CZ', 'DE', 'DK', 'EE', 'ES', 'FI', 'FR', 'GB', 'GR', 'HR', 'HU', 'IE', 'IT', 'LT', 'LU', 'LV', 'ME', 'MK', 'NL', 'NO', 'PL', 'PT', 'RO', 'RS', 'SE', 'SI', 'SK']

snapshots:
 # start: "2013-01-01"
  start: "2016-01-17"
  end: "2016-01-23"
  inclusive: 'left' # include start, not end

enable:
  prepare_links_p_nom: false
  retrieve_databundle: true
  retrieve_cost_data: true
  build_cutout: true
  #build_cutout: false
  retrieve_cutout: false
  #retrieve_cutout: true
  build_natura_raster: false
  retrieve_natura_raster: true
  custom_busmap: false

electricity:
  voltages: [220., 300., 380.]
  gaslimit: false # global gas usage limit of X MWh_th
  co2limit: 7.75e+7 # 0.05 * 3.1e9*0.5
  co2base: 1.487e+9
  agg_p_nom_limits: data/agg_p_nom_minmax.csv

  operational_reserve: # like https://genxproject.github.io/GenX/dev/core/#Reserves
    activate: false
    epsilon_load: 0.02 # share of total load
    epsilon_vres: 0.02 # share of total renewable supply
    contingency: 4000 # fixed capacity in MW

  max_hours:
    battery: 6
    H2: 168

  extendable_carriers:
    Generator: []
  #  Generator: [solar, onwind, offwind-ac, offwind-dc, OCGT]
    StorageUnit: [battery] # battery, H2
    Store: [battery, H2]
    Link: [] # H2 pipeline

  # use pandas query strings here, e.g. Country not in ['Germany']
  powerplants_filter: (DateOut >= 2022 or DateOut != DateOut)
  # use pandas query strings here, e.g. Country in ['Germany']
  custom_powerplants: false

  conventional_carriers: [nuclear, oil, OCGT, CCGT, coal, lignite, geothermal, biomass]
  renewable_carriers: [solar, onwind, offwind-ac, offwind-dc, hydro]

  estimate_renewable_capacities:
    enable: true
    # Add capacities from OPSD data
    from_opsd: true
    # Renewable capacities are based on existing capacities reported by IRENA
    year: 2020
    # Artificially limit maximum capacities to factor * (IRENA capacities),
    # i.e. 110% of <years>'s capacities => expansion_limit: 1.1
    # false: Use estimated renewable potentials determine by the workflow
    expansion_limit: false
    technology_mapping:
      # Wind is the Fueltype in powerplantmatching, onwind, offwind-{ac,dc} the carrier in PyPSA-Eur
      Offshore: [offwind-ac, offwind-dc]
      Onshore: [onwind]
      PV: [solar]

atlite:
  nprocesses: 4
  show_progress: false # false saves time
  cutouts:
    # use 'base' to determine geographical bounds and time span from config
    # base:
      # module: era5
    europe-2013-era5:
      module: era5 # in priority order
      x: [-12., 35.]
      y: [33., 72]
      dx: 0.3
      dy: 0.3
      time: ['2016-01-01', '2016-02-01']
      #time: ['2013', '2013']
    europe-2013-sarah:
      module: [sarah, era5] # in priority order
      x: [-12., 45.]
      y: [33., 65]
      dx: 0.2
      dy: 0.2
      time: ['2016-01-01', '2016-02-01']
     # time: ['2013', '2013']
      sarah_interpolate: false
      sarah_dir:
      features: [influx, temperature]


renewable:
  onwind:
    cutout: europe-2013-era5
    resource:
      method: wind
      turbine: Vestas_V112_3MW
    capacity_per_sqkm: 3 # ScholzPhd Tab 4.3.1: 10MW/km^2 and assuming 30% fraction of the already restricted
    # area is available for installation of wind generators due to competing land use and likely public
    # acceptance issues.
    # correction_factor: 0.93
    corine:
      # Scholz, Y. (2012). Renewable energy based electricity supply at low costs:
      #  development of the REMix model and application for Europe. ( p.42 / p.28)
      grid_codes: [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 32]
      distance: 1000
      distance_grid_codes: [1, 2, 3, 4, 5, 6]
    natura: true
    excluder_resolution: 100
    potential: simple # or conservative
    clip_p_max_pu: 1.e-2
  offwind-ac:
    cutout: europe-2013-era5
    resource:
      method: wind
      turbine: NREL_ReferenceTurbine_5MW_offshore
    capacity_per_sqkm: 2 # ScholzPhd Tab 4.3.1: 10MW/km^2 and assuming 20% fraction of the already restricted
    # area is available for installation of wind generators due to competing land use and likely public
    # acceptance issues.
    correction_factor: 0.8855
    # proxy for wake losses
    # from 10.1016/j.energy.2018.08.153
    # until done more rigorously in #153
    corine: [44, 255]
    natura: true
    ship_threshold: 400
    max_depth: 50
    max_shore_distance: 30000
    excluder_resolution: 200
    potential: simple # or conservative
    clip_p_max_pu: 1.e-2
  offwind-dc:
    cutout: europe-2013-era5
    resource:
      method: wind
      turbine: NREL_ReferenceTurbine_5MW_offshore
    capacity_per_sqkm: 2 # ScholzPhd Tab 4.3.1: 10MW/km^2 and assuming 20% fraction of the already restricted
    # area is available for installation of wind generators due to competing land use and likely public
    # acceptance issues.
    correction_factor: 0.8855
    # proxy for wake losses
    # from 10.1016/j.energy.2018.08.153
    # until done more rigorously in #153
    corine: [44, 255]
    natura: true
    ship_threshold: 400
    max_depth: 50
    min_shore_distance: 30000
    excluder_resolution: 200
    potential: simple # or conservative
    clip_p_max_pu: 1.e-2
  solar:
    cutout: europe-2013-sarah
    resource:
      method: pv
      panel: CSi
      orientation:
        slope: 35.
        azimuth: 180.
    capacity_per_sqkm: 1.7 # ScholzPhd Tab 4.3.1: 170 MW/km^2 and assuming 1% of the area can be used for solar PV panels
    # Correction factor determined by comparing uncorrected area-weighted full-load hours to those
    # published in Supplementary Data to
    # Pietzcker, Robert Carl, et al. "Using the sun to decarbonize the power
    # sector: The economic potential of photovoltaics and concentrating solar
    # power." Applied Energy 135 (2014): 704-720.
    # This correction factor of 0.854337 may be in order if using reanalysis data.
    # for discussion refer to https://github.com/PyPSA/pypsa-eur/pull/304
    # correction_factor: 0.854337
    corine: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 26, 31, 32]
    natura: true
    excluder_resolution: 100
    potential: simple # or conservative
    clip_p_max_pu: 1.e-2
  hydro:
    cutout: europe-2013-era5
    carriers: [ror, PHS, hydro]
    PHS_max_hours: 6
    hydro_max_hours: "energy_capacity_totals_by_country" # one of energy_capacity_totals_by_country, estimate_by_large_installations or a float
    clip_min_inflow: 1.0

conventional:
  nuclear:
    p_max_pu: "data/nuclear_p_max_pu.csv" # float of file name

lines:
  types:
    220.: "Al/St 240/40 2-bundle 220.0"
    300.: "Al/St 240/40 3-bundle 300.0"
    380.: "Al/St 240/40 4-bundle 380.0"
  s_max_pu: 0.7
  s_nom_max: .inf
  length_factor: 1.25
  under_construction: 'zero' # 'zero': set capacity to zero, 'remove': remove, 'keep': with full capacity

links:
  p_max_pu: 1.0
  p_nom_max: .inf
  include_tyndp: true
  under_construction: 'zero' # 'zero': set capacity to zero, 'remove': remove, 'keep': with full capacity

transformers:
  x: 0.1
  s_nom: 2000.
  type: ''

load:
  power_statistics: true # only for files from <2019; set false in order to get ENTSOE transparency data
  interpolate_limit: 3 # data gaps up until this size are interpolated linearly
  time_shift_for_large_gaps: 1w # data gaps up until this size are copied by copying from
  manual_adjustments: true # false
  scaling_factor: 1.0

costs:
  year: 2030
  version: v0.5.0
  rooftop_share: 0.14  # based on the potentials, assuming  (0.1 kW/m2 and 10 m2/person)
  fill_values:
    FOM: 0
    VOM: 0
    efficiency: 1
    fuel: 0
    investment: 0
    lifetime: 25
    "CO2 intensity": 0
    "discount rate": 0.07
  marginal_cost:
    solar: 0.01
    onwind: 0.015
    offwind: 0.015
    hydro: 0.
    H2: 0.
    electrolysis: 0.
    fuel cell: 0.
    battery: 0.
    battery inverter: 0.
  emission_prices: # in currency per tonne emission, only used with the option Ep
    co2: 0.

clustering:
  simplify_network:
    to_substations: false # network is simplified to nodes with positive or negative power injection (i.e. substations or offwind connections)
    algorithm: kmeans # choose from: [hac, kmeans]
    feature: solar+onwind-time # only for hac. choose from: [solar+onwind-time, solar+onwind-cap, solar-time, solar-cap, solar+offwind-cap] etc.
    exclude_carriers: []
    remove_stubs: true
    remove_stubs_across_borders: true
  cluster_network:
    algorithm: kmeans
    feature: solar+onwind-time
    exclude_carriers: []
  aggregation_strategies:
    generators:
      p_nom_max: sum # use "min" for more conservative assumptions
      p_nom_min: sum
      p_min_pu: mean
      marginal_cost: mean
      committable: any
      ramp_limit_up: max
      ramp_limit_down: max
      efficiency: mean

solving:
  options:
    formulation: kirchhoff
    load_shedding: false
    noisy_costs: true
    min_iterations: 4
    max_iterations: 6
    clip_p_max_pu: 0.01
    skip_iterations: true
    track_iterations: false
    #nhours: 10
  solver:
    name: cbc
   # threads: 4
   # method: 2 # barrier
   # crossover: 0
   # BarConvTol: 1.e-5
   # FeasibilityTol: 1.e-6
   # AggFill: 0
   # PreDual: 0
   # GURO_PAR_BARDENSETHRESH: 200
  # solver:
  #   name: cplex
  #   threads: 4
  #   lpmethod: 4 # barrier
  #   solutiontype: 2 # non basic solution, ie no crossover
  #   barrier.convergetol: 1.e-5
  #   feasopt.tolerance: 1.e-6

plotting:
  map:
    figsize: [7, 7]
    boundaries: [-10.2, 29, 35, 72]
    p_nom:
      bus_size_factor: 5.e+4
      linewidth_factor: 3.e+3

  costs_max: 800
  costs_threshold: 1

  energy_max: 15000.
  energy_min: -10000.
  energy_threshold: 50.

  vre_techs: ["onwind", "offwind-ac", "offwind-dc", "solar", "ror"]
  conv_techs: ["OCGT", "CCGT", "Nuclear", "Coal"]
  storage_techs: ["hydro+PHS", "battery", "H2"]
  load_carriers: ["AC load"]
  AC_carriers: ["AC line", "AC transformer"]
  link_carriers: ["DC line", "Converter AC-DC"]
  tech_colors:
    "onwind": "#235ebc"
    "onshore wind": "#235ebc"
    'offwind': "#6895dd"
    'offwind-ac': "#6895dd"
    'offshore wind': "#6895dd"
    'offshore wind ac': "#6895dd"
    'offwind-dc': "#74c6f2"
    'offshore wind dc': "#74c6f2"
    "hydro": "#08ad97"
    "hydro+PHS": "#08ad97"
    "PHS": "#08ad97"
    "hydro reservoir": "#08ad97"
    'hydroelectricity': '#08ad97'
    "ror": "#4adbc8"
    "run of river": "#4adbc8"
    'solar': "#f9d002"
    'solar PV': "#f9d002"
    'solar thermal': '#ffef60'
    'biomass': '#0c6013'
    'solid biomass': '#06540d'
    'biogas': '#23932d'
    'waste': '#68896b'
    'geothermal': '#ba91b1'
    "OCGT": "#d35050"
    "gas": "#d35050"
    "natural gas": "#d35050"
    "CCGT": "#b20101"
    "nuclear": "#ff9000"
    "coal": "#707070"
    "lignite": "#9e5a01"
    "oil": "#262626"
    "H2": "#ea048a"
    "hydrogen storage": "#ea048a"
    "battery": "#b8ea04"
    "Electric load": "#f9d002"
    "electricity": "#f9d002"
    "lines": "#70af1d"
    "transmission lines": "#70af1d"
    "AC-AC": "#70af1d"
    "AC line": "#70af1d"
    "links": "#8a1caf"
    "HVDC links": "#8a1caf"
    "DC-DC": "#8a1caf"
    "DC link": "#8a1caf"
  nice_names:
    OCGT: "Open-Cycle Gas"
    CCGT: "Combined-Cycle Gas"
    offwind-ac: "Offshore Wind (AC)"
    offwind-dc: "Offshore Wind (DC)"
    onwind: "Onshore Wind"
    solar: "Solar"
    PHS: "Pumped Hydro Storage"
    hydro: "Reservoir & Dam"
    battery: "Battery Storage"
    H2: "Hydrogen Storage"
    lines: "Transmission Lines"
    ror: "Run of River"

Your Environment

I'm using PyPSA-eur v0.7.0

The atlite version used:
How you installed atlite (conda, pip or github):
Operating System:
My environment:

(output of `conda list`)
```

packages in environment at /home/ubuntu/anaconda3/envs/pypsa-eur:

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
affine 2.4.0 pyhd8ed1ab_0 conda-forge
alsa-lib 1.2.8 h166bdaf_0 conda-forge
ampl-mp 3.1.0 h2cc385e_1006 conda-forge
amply 0.1.5 pyhd8ed1ab_0 conda-forge
appdirs 1.4.4 pyh9f0ad1d_0 conda-forge
arrow-cpp 11.0.0 ha770c72_13_cpu conda-forge
asttokens 2.2.1 pyhd8ed1ab_0 conda-forge
atlite 0.2.10 pyhd8ed1ab_0 conda-forge
attr 2.5.1 h166bdaf_1 conda-forge
attrs 22.2.0 pyh71513ae_0 conda-forge
aws-c-auth 0.6.26 hf365957_1 conda-forge
aws-c-cal 0.5.21 h48707d8_2 conda-forge
aws-c-common 0.8.14 h0b41bf4_0 conda-forge
aws-c-compression 0.2.16 h03acc5a_5 conda-forge
aws-c-event-stream 0.2.20 h00877a2_4 conda-forge
aws-c-http 0.7.6 hf342b9f_0 conda-forge
aws-c-io 0.13.19 h5b20300_3 conda-forge
aws-c-mqtt 0.8.6 hc4349f7_12 conda-forge
aws-c-s3 0.2.7 h909e904_1 conda-forge
aws-c-sdkutils 0.1.8 h03acc5a_0 conda-forge
aws-checksums 0.1.14 h03acc5a_5 conda-forge
aws-crt-cpp 0.19.8 hf7fbfca_12 conda-forge
aws-sdk-cpp 1.10.57 h17c43bd_8 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 pyhd8ed1ab_3 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.12.0 pyha770c72_0 conda-forge
blosc 1.21.3 hafa529b_0 conda-forge
bokeh 2.4.3 pyhd8ed1ab_3 conda-forge
boost-cpp 1.78.0 h75c5d50_1 conda-forge
bottleneck 1.3.7 py310h0a54255_0 conda-forge
branca 0.6.0 pyhd8ed1ab_0 conda-forge
brotli 1.0.9 h166bdaf_8 conda-forge
brotli-bin 1.0.9 h166bdaf_8 conda-forge
brotlipy 0.7.0 py310h5764c6d_1005 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.18.1 h7f98852_0 conda-forge
ca-certificates 2022.12.7 ha878542_0 conda-forge
cairo 1.16.0 ha61ee94_1014 conda-forge
cartopy 0.21.1 py310h7eb24ba_1 conda-forge
cdsapi 0.6.1 pyhd8ed1ab_0 conda-forge
certifi 2022.12.7 pyhd8ed1ab_0 conda-forge
cffi 1.15.1 py310h255011f_3 conda-forge
cfitsio 4.2.0 hd9d235c_0 conda-forge
cftime 1.6.2 py310hde88566_1 conda-forge
charset-normalizer 3.1.0 pyhd8ed1ab_0 conda-forge
click 8.1.3 unix_pyhd8ed1ab_2 conda-forge
click-plugins 1.1.1 py_0 conda-forge
cligj 0.7.2 pyhd8ed1ab_1 conda-forge
cloudpickle 2.2.1 pyhd8ed1ab_0 conda-forge
coin-or-cbc 2.10.8 h3786ebc_0 conda-forge
coin-or-cgl 0.60.6 h6f57e76_2 conda-forge
coin-or-clp 1.17.7 hc56784d_2 conda-forge
coin-or-osi 0.108.7 h2720bb7_2 conda-forge
coin-or-utils 2.11.6 h202d8b1_2 conda-forge
coincbc 2.10.8 0_metapackage conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
configargparse 1.5.3 pyhd8ed1ab_0 conda-forge
connection_pool 0.0.3 pyhd3deb0d_0 conda-forge
country_converter 1.0.0 pyhd8ed1ab_1 conda-forge
countrycode 0.2 pypi_0 pypi
cryptography 40.0.1 py310h34c0648_0 conda-forge
curl 7.88.1 hdc1c0ab_1 conda-forge
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
cytoolz 0.12.0 py310h5764c6d_1 conda-forge
dask 2023.3.2 pyhd8ed1ab_0 conda-forge
dask-core 2023.3.2 pyhd8ed1ab_0 conda-forge
datrie 0.8.2 py310h5764c6d_6 conda-forge
dbus 1.13.6 h5008d03_3 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
deprecation 2.1.0 pyh9f0ad1d_0 conda-forge
descartes 1.1.0 py_4 conda-forge
distributed 2023.3.2 pyhd8ed1ab_0 conda-forge
distro 1.8.0 pyhd8ed1ab_0 conda-forge
docutils 0.19 py310hff52083_1 conda-forge
dpath 2.1.5 py310hff52083_0 conda-forge
entsoe-py 0.5.8 pyhd8ed1ab_0 conda-forge
et_xmlfile 1.1.0 pyhd8ed1ab_0 conda-forge
exceptiongroup 1.1.1 pyhd8ed1ab_0 conda-forge
executing 1.2.0 pyhd8ed1ab_0 conda-forge
expat 2.5.0 hcb278e6_1 conda-forge
fftw 3.3.10 nompi_hf0379b8_106 conda-forge
filelock 3.10.7 pyhd8ed1ab_0 conda-forge
fiona 1.9.2 py310ha325b7b_0 conda-forge
folium 0.14.0 pyhd8ed1ab_0 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.39.3 py310h1fa729e_0 conda-forge
freetype 2.12.1 hca18f0e_1 conda-forge
freexl 1.0.6 h166bdaf_1 conda-forge
fsspec 2023.3.0 pyhd8ed1ab_1 conda-forge
gdal 3.6.3 py310hc1b7723_1 conda-forge
geographiclib 1.52 pyhd8ed1ab_0 conda-forge
geojson-rewind 1.0.2 pyhd8ed1ab_0 conda-forge
geopandas 0.12.2 pyhd8ed1ab_0 conda-forge
geopandas-base 0.12.2 pyha770c72_0 conda-forge
geopy 2.3.0 pyhd8ed1ab_0 conda-forge
geos 3.11.2 hcb278e6_0 conda-forge
geotiff 1.7.1 hb963b44_7 conda-forge
gettext 0.21.1 h27087fc_0 conda-forge
gflags 2.2.2 he1b5a44_1004 conda-forge
giflib 5.2.1 h0b41bf4_3 conda-forge
gitdb 4.0.10 pyhd8ed1ab_0 conda-forge
gitpython 3.1.31 pyhd8ed1ab_0 conda-forge
glib 2.74.1 h6239696_1 conda-forge
glib-tools 2.74.1 h6239696_1 conda-forge
glog 0.6.0 h6f12383_0 conda-forge
graphite2 1.3.13 h58526e2_1001 conda-forge
gst-plugins-base 1.22.0 h4243ec0_2 conda-forge
gstreamer 1.22.0 h25f0c4b_2 conda-forge
gstreamer-orc 0.4.33 h166bdaf_0 conda-forge
harfbuzz 6.0.0 h8e241bc_0 conda-forge
hdf4 4.2.15 h501b40f_6 conda-forge
hdf5 1.12.2 nompi_h4df4325_101 conda-forge
heapdict 1.0.1 py_0 conda-forge
highspy 1.5.0.dev0 pypi_0 pypi
humanfriendly 10.0 py310hff52083_4 conda-forge
icu 70.1 h27087fc_0 conda-forge
idna 3.4 pyhd8ed1ab_0 conda-forge
importlib-metadata 6.1.0 pyha770c72_0 conda-forge
importlib_metadata 6.1.0 hd8ed1ab_0 conda-forge
importlib_resources 5.12.0 pyhd8ed1ab_0 conda-forge
iniconfig 2.0.0 pyhd8ed1ab_0 conda-forge
ipopt 3.14.11 hf9e1ecf_0 conda-forge
ipython 8.12.0 pyh41d4057_0 conda-forge
jack 1.9.22 h11f4161_0 conda-forge
jedi 0.18.2 pyhd8ed1ab_0 conda-forge
jinja2 3.1.2 pyhd8ed1ab_1 conda-forge
joblib 1.2.0 pyhd8ed1ab_0 conda-forge
json-c 0.16 hc379101_0 conda-forge
jsonschema 4.17.3 pyhd8ed1ab_0 conda-forge
jupyter_core 5.3.0 py310hff52083_0 conda-forge
kealib 1.5.0 ha7026e8_0 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.4 py310hbf28c38_1 conda-forge
krb5 1.20.1 h81ceb04_0 conda-forge
lame 3.100 h166bdaf_1003 conda-forge
lcms2 2.15 haa2dc70_1 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libabseil 20230125.0 cxx17_hcb278e6_1 conda-forge
libaec 1.0.6 hcb278e6_1 conda-forge
libarrow 11.0.0 h93537a5_13_cpu conda-forge
libblas 3.9.0 16_linux64_openblas conda-forge
libbrotlicommon 1.0.9 h166bdaf_8 conda-forge
libbrotlidec 1.0.9 h166bdaf_8 conda-forge
libbrotlienc 1.0.9 h166bdaf_8 conda-forge
libcap 2.67 he9d0100_0 conda-forge
libcblas 3.9.0 16_linux64_openblas conda-forge
libclang 15.0.7 default_had23c3d_1 conda-forge
libclang13 15.0.7 default_h3e3d535_1 conda-forge
libcrc32c 1.1.2 h9c3ff4c_0 conda-forge
libcups 2.3.3 h36d4200_3 conda-forge
libcurl 7.88.1 hdc1c0ab_1 conda-forge
libdb 6.2.32 h9c3ff4c_0 conda-forge
libdeflate 1.17 h0b41bf4_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libevent 2.1.10 h28343ad_4 conda-forge
libexpat 2.5.0 hcb278e6_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libflac 1.4.2 h27087fc_0 conda-forge
libgcc-ng 12.2.0 h65d4601_19 conda-forge
libgcrypt 1.10.1 h166bdaf_0 conda-forge
libgdal 3.6.3 h93ed92d_1 conda-forge
libgfortran-ng 12.2.0 h69a702a_19 conda-forge
libgfortran5 12.2.0 h337968e_19 conda-forge
libglib 2.74.1 h606061b_1 conda-forge
libgomp 12.2.0 h65d4601_19 conda-forge
libgoogle-cloud 2.8.0 h0bc5f78_1 conda-forge
libgpg-error 1.46 h620e276_0 conda-forge
libgrpc 1.52.1 hcf146ea_1 conda-forge
libiconv 1.17 h166bdaf_0 conda-forge
libjpeg-turbo 2.1.5.1 h0b41bf4_0 conda-forge
libkml 1.3.0 h37653c0_1015 conda-forge
liblapack 3.9.0 16_linux64_openblas conda-forge
liblapacke 3.9.0 16_linux64_openblas conda-forge
libllvm15 15.0.7 hadd5161_1 conda-forge
libnetcdf 4.9.1 nompi_hd2e9713_102 conda-forge
libnghttp2 1.52.0 h61bc06f_0 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libnuma 2.0.16 h0b41bf4_1 conda-forge
libogg 1.3.4 h7f98852_1 conda-forge
libopenblas 0.3.21 pthreads_h78a6416_3 conda-forge
libopus 1.3.1 h7f98852_1 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
libpq 15.2 hb675445_0 conda-forge
libprotobuf 3.21.12 h3eb15da_0 conda-forge
librttopo 1.1.0 h0d5128d_13 conda-forge
libsndfile 1.2.0 hb75c966_0 conda-forge
libspatialindex 1.9.3 h9c3ff4c_4 conda-forge
libspatialite 5.0.1 h2d2bb50_24 conda-forge
libsqlite 3.40.0 h753d276_0 conda-forge
libssh2 1.10.0 hf14f497_3 conda-forge
libstdcxx-ng 12.2.0 h46fd767_19 conda-forge
libsystemd0 253 h8c4010b_1 conda-forge
libthrift 0.18.1 h5e4af38_0 conda-forge
libtiff 4.5.0 hddfeb54_5 conda-forge
libtool 2.4.7 h27087fc_0 conda-forge
libudev1 253 h0b41bf4_1 conda-forge
libutf8proc 2.8.0 h166bdaf_0 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libvorbis 1.3.7 h9c3ff4c_0 conda-forge
libwebp-base 1.3.0 h0b41bf4_0 conda-forge
libxcb 1.13 h7f98852_1004 conda-forge
libxkbcommon 1.5.0 h79f4944_1 conda-forge
libxml2 2.10.3 hca2bb57_4 conda-forge
libxslt 1.1.37 h873f0b0_0 conda-forge
libzip 1.9.2 hc929e4a_1 conda-forge
libzlib 1.2.13 h166bdaf_4 conda-forge
linopy 0.1.4 pyhd8ed1ab_0 conda-forge
locket 1.0.0 pyhd8ed1ab_0 conda-forge
lxml 4.9.2 py310hbdc0903_0 conda-forge
lz4 4.3.2 py310h0cfdcf0_0 conda-forge
lz4-c 1.9.4 hcb278e6_0 conda-forge
lzo 2.10 h516909a_1000 conda-forge
mapclassify 2.5.0 pyhd8ed1ab_1 conda-forge
markupsafe 2.1.2 py310h1fa729e_0 conda-forge
matplotlib 3.5.3 py310hff52083_2 conda-forge
matplotlib-base 3.5.3 py310h8d5ebf3_2 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
memory_profiler 0.61.0 pyhd8ed1ab_0 conda-forge
metis 5.1.0 h58526e2_1006 conda-forge
mpg123 1.31.3 hcb278e6_0 conda-forge
msgpack-python 1.0.5 py310hdf3cbec_0 conda-forge
mumps-include 5.2.1 ha770c72_11 conda-forge
mumps-seq 5.2.1 h2104b81_11 conda-forge
munch 2.5.0 py_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
mysql-common 8.0.32 ha901b37_1 conda-forge
mysql-libs 8.0.32 hd7da12d_1 conda-forge
nbformat 5.8.0 pyhd8ed1ab_0 conda-forge
ncurses 6.3 h27087fc_1 conda-forge
netcdf4 1.6.3 nompi_py310h0feb132_100 conda-forge
networkx 3.0 pyhd8ed1ab_0 conda-forge
nomkl 1.0 h5ca1d4c_0 conda-forge
nspr 4.35 h27087fc_0 conda-forge
nss 3.89 he45b914_0 conda-forge
numexpr 2.8.3 py310hf05e7a9_101 conda-forge
numpy 1.23.5 py310h53a5b5f_0 conda-forge
openjdk 17.0.3 h4335b31_6 conda-forge
openjpeg 2.5.0 hfec8fc6_2 conda-forge
openpyxl 3.1.1 py310h1fa729e_0 conda-forge
openssl 3.1.0 h0b41bf4_0 conda-forge
orc 1.8.3 hfdbbad2_0 conda-forge
packaging 23.0 pyhd8ed1ab_0 conda-forge
pandas 1.5.3 py310h9b08913_1 conda-forge
parquet-cpp 1.5.1 2 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
partd 1.3.0 pyhd8ed1ab_0 conda-forge
patsy 0.5.3 pyhd8ed1ab_0 conda-forge
pcre2 10.40 hc3806b6_0 conda-forge
pexpect 4.8.0 pyh1a96a4e_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 9.4.0 py310h065c6d2_2 conda-forge
pip 23.0.1 pyhd8ed1ab_0 conda-forge
pixman 0.40.0 h36c2ea0_0 conda-forge
pkgutil-resolve-name 1.3.10 pyhd8ed1ab_0 conda-forge
plac 1.3.5 pyhd8ed1ab_0 conda-forge
platformdirs 3.2.0 pyhd8ed1ab_0 conda-forge
pluggy 1.0.0 pyhd8ed1ab_5 conda-forge
ply 3.11 py_1 conda-forge
pooch 1.7.0 pyha770c72_3 conda-forge
poppler 23.03.0 hf052cbe_1 conda-forge
poppler-data 0.4.12 hd8ed1ab_0 conda-forge
postgresql 15.2 h3248436_0 conda-forge
powerplantmatching 0.5.6 pyhd8ed1ab_0 conda-forge
progressbar2 4.2.0 pyhd8ed1ab_0 conda-forge
proj 9.1.1 h8ffa02c_2 conda-forge
prompt-toolkit 3.0.38 pyha770c72_0 conda-forge
prompt_toolkit 3.0.38 hd8ed1ab_0 conda-forge
psutil 5.9.4 py310h5764c6d_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pulp 2.7.0 py310hff52083_0 conda-forge
pulseaudio 16.1 hcb278e6_3 conda-forge
pulseaudio-client 16.1 h5195f5e_3 conda-forge
pulseaudio-daemon 16.1 ha8d29e2_3 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pyarrow 11.0.0 py310h633f555_13_cpu conda-forge
pybind11 2.10.4 pypi_0 pypi
pycountry 22.3.5 pyhd8ed1ab_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.14.0 pyhd8ed1ab_0 conda-forge
pyomo 6.5.0 py310heca2aa9_0 conda-forge
pyopenssl 23.1.1 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pyproj 3.5.0 py310h15e2413_0 conda-forge
pypsa 0.22.1 pyhd8ed1ab_0 conda-forge
pyqt 5.15.7 py310hab646b1_3 conda-forge
pyqt5-sip 12.11.0 py310heca2aa9_3 conda-forge
pyrsistent 0.19.3 py310h1fa729e_0 conda-forge
pyshp 2.3.1 pyhd8ed1ab_0 conda-forge
pysocks 1.7.1 pyha2e5f31_6 conda-forge
pytables 3.7.0 py310hb60b9b2_3 conda-forge
pytest 7.2.2 pyhd8ed1ab_0 conda-forge
python 3.10.10 he550d4f_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-fastjsonschema 2.16.3 pyhd8ed1ab_0 conda-forge
python-utils 3.5.2 pyhd8ed1ab_0 conda-forge
python_abi 3.10 3_cp310 conda-forge
pytz 2023.3 pyhd8ed1ab_0 conda-forge
pyxlsb 1.0.10 pyhd8ed1ab_0 conda-forge
pyyaml 6.0 py310h5764c6d_5 conda-forge
qt-main 5.15.8 h67dfc38_7 conda-forge
rasterio 1.3.6 py310h3e853a9_0 conda-forge
re2 2023.02.02 hcb278e6_0 conda-forge
readline 8.2 h8228510_1 conda-forge
requests 2.28.2 pyhd8ed1ab_1 conda-forge
reretry 0.11.8 pyhd8ed1ab_0 conda-forge
rtree 1.0.1 py310hbdcdc62_1 conda-forge
s2n 1.3.41 h3358134_0 conda-forge
scikit-learn 1.2.2 py310h41b6a48_1 conda-forge
scipy 1.10.1 py310h8deb116_0 conda-forge
scotch 6.0.9 hb2e6521_2 conda-forge
seaborn 0.12.2 hd8ed1ab_0 conda-forge
seaborn-base 0.12.2 pyhd8ed1ab_0 conda-forge
setuptools 67.6.1 pyhd8ed1ab_0 conda-forge
setuptools-scm 7.1.0 pyhd8ed1ab_0 conda-forge
setuptools_scm 7.1.0 hd8ed1ab_0 conda-forge
shapely 2.0.1 py310h056c13c_1 conda-forge
sip 6.7.7 py310heca2aa9_1 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
smart_open 6.3.0 pyhd8ed1ab_1 conda-forge
smmap 3.0.5 pyh44b312d_0 conda-forge
snakemake-minimal 7.25.0 pyhdfd78af_0 bioconda
snappy 1.1.10 h9fff704_0 conda-forge
snuggs 1.4.7 py_0 conda-forge
sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge
soupsieve 2.3.2.post1 pyhd8ed1ab_0 conda-forge
sqlite 3.40.0 h4ff8645_0 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
statsmodels 0.13.5 py310hde88566_2 conda-forge
stopit 1.1.2 py_0 conda-forge
tabula-py 2.6.0 py310hff52083_0 conda-forge
tabulate 0.9.0 pyhd8ed1ab_1 conda-forge
tblib 1.7.0 pyhd8ed1ab_0 conda-forge
threadpoolctl 3.1.0 pyh8a188c0_0 conda-forge
throttler 1.2.1 pyhd8ed1ab_0 conda-forge
tiledb 2.13.2 hd532e3d_0 conda-forge
tk 8.6.12 h27826a3_0 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
toolz 0.12.0 pyhd8ed1ab_0 conda-forge
toposort 1.10 pyhd8ed1ab_0 conda-forge
tornado 6.2 py310h5764c6d_1 conda-forge
tqdm 4.65.0 pyhd8ed1ab_1 conda-forge
traitlets 5.9.0 pyhd8ed1ab_0 conda-forge
tsam 2.2.2 pypi_0 pypi
typing-extensions 4.5.0 hd8ed1ab_0 conda-forge
typing_extensions 4.5.0 pyha770c72_0 conda-forge
tzcode 2023c h0b41bf4_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
ucx 1.14.0 ha0ee010_0 conda-forge
unicodedata2 15.0.0 py310h5764c6d_0 conda-forge
unidecode 1.3.6 pyhd8ed1ab_0 conda-forge
unixodbc 2.3.10 h583eb01_0 conda-forge
urllib3 1.26.15 pyhd8ed1ab_0 conda-forge
vresutils 0.3.1 pypi_0 pypi
wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge
wheel 0.40.0 pyhd8ed1ab_0 conda-forge
wrapt 1.15.0 py310h1fa729e_0 conda-forge
xarray 2023.3.0 pyhd8ed1ab_0 conda-forge
xcb-util 0.4.0 h166bdaf_0 conda-forge
xcb-util-image 0.4.0 h166bdaf_0 conda-forge
xcb-util-keysyms 0.4.0 h166bdaf_0 conda-forge
xcb-util-renderutil 0.3.9 h166bdaf_0 conda-forge
xcb-util-wm 0.4.1 h166bdaf_0 conda-forge
xerces-c 3.2.4 h55805fa_1 conda-forge
xkeyboard-config 2.38 h0b41bf4_0 conda-forge
xlrd 2.0.1 pyhd8ed1ab_3 conda-forge
xorg-fixesproto 5.0 h7f98852_1002 conda-forge
xorg-inputproto 2.3.2 h7f98852_1002 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.0.10 h7f98852_0 conda-forge
xorg-libsm 1.2.3 hd9c2040_1000 conda-forge
xorg-libx11 1.8.4 h0b41bf4_0 conda-forge
xorg-libxau 1.0.9 h7f98852_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxfixes 5.0.3 h7f98852_1004 conda-forge
xorg-libxi 1.7.10 h7f98852_0 conda-forge
xorg-libxrender 0.9.10 h7f98852_1003 conda-forge
xorg-libxtst 1.2.3 h7f98852_1002 conda-forge
xorg-recordproto 1.14.2 h7f98852_1002 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
xorg-xf86vidmodeproto 2.3.1 h7f98852_1002 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xyzservices 2023.2.0 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
yaml 0.2.5 h7f98852_2 conda-forge
yte 1.5.1 py310hff52083_1 conda-forge
zict 2.2.0 pyhd8ed1ab_0 conda-forge
zipp 3.15.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 h166bdaf_4 conda-forge
zstd 1.5.2 h3eb15da_6 conda-forge

    <!-- output of `conda list` -->
  ```
</details>

Thank you!

The text was updated successfully, but these errors were encountered:

pz-max · 2023-04-18T08:24:37Z

@Rock910, creating new cutouts usually needs some extra attention. Did this Sarah example work for you? I can also recommend testing the ERA5 cutout if you haven't tried yet.

euronion · 2023-04-19T09:36:56Z

Hi @Rock910 ,

Building the cutouts is a tedious task. For ERA5 the data can be downloaded automatically as long as you have registered with the CDSAPI. For SARAH data (and creation of the SARAH cutout), you have to download the raw SARAH data first and pass the raw data folder to PyPSA-EUR via this line in the config.yaml:

      sarah_dir:

Right now it is empty (in your config above). Empty means NULL, and that creates the error you see.

Rock910 · 2023-05-01T17:34:42Z

Thank you for your reply, would there be any way to have a prebuilt SARAH cutout for the year 2016 instead? I was trying to get the data but it looks like the total size is 7.5 TiB.

euronion · 2023-05-01T17:56:12Z

I can't offer you a prebuilt cutout for 2016.

The SARAH data should not be 7.5 TiB, you might have selected the wrong product or maybe you did not limit the temporal range for which to download the raw data.

If you select the right product and limit the temporal range for which you download the variable, you end up with a more reasonable 98 GiB of size for the raw data, see here:

euronion · 2023-08-10T07:10:26Z

Hi @Rock910

Did you have success with downloading and building a SARAH cutout?

Rock910 added status: needs classification type: bug labels Apr 17, 2023

ekatef mentioned this issue Apr 21, 2023

Add SARAH-2 data to build_cutout script pypsa-meets-earth/pypsa-earth#74

Open

euronion added type: discussion and removed type: bug status: needs classification labels Aug 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue with build_cutout using alite #286

Issue with build_cutout using alite #286

Rock910 commented Apr 17, 2023 •

edited

pz-max commented Apr 18, 2023

euronion commented Apr 19, 2023

Rock910 commented May 1, 2023

euronion commented May 1, 2023

euronion commented Aug 10, 2023

Issue with build_cutout using alite #286

Issue with build_cutout using alite #286

Comments

Rock910 commented Apr 17, 2023 • edited

Your Environment

packages in environment at /home/ubuntu/anaconda3/envs/pypsa-eur:

Name Version Build Channel

pz-max commented Apr 18, 2023

euronion commented Apr 19, 2023

Rock910 commented May 1, 2023

euronion commented May 1, 2023

euronion commented Aug 10, 2023

Rock910 commented Apr 17, 2023 •

edited