Production server (rees-web04) setup and software version

A brief guide to the production server state, software, data, and issues

Server configuration

For the API release, a basic configuration was implemented on the production server. The steps were:

Installed nginx and made the pvgis6.conf file available at /etc/nginx/sites-available/pvgis6.conf ▶️ linked to /etc/nginx/sites-enabled/
Prepared the *.service file (located at /etc/systemd/system/pvgis6.service). To start and stop the service:
- sudo systemctl start pvgis6
- sudo systemctl stop pvgis6
Additional steps to expose the web API to the WELCOME network were performed by C4; no further details were provided to the PVGIS team.

Data processing

This server was used for data preparation (the dedicated repository can be found here), specifically to create new time series from 2014 to 2024. The meteorological variables prepared were:

Direct horizontal irradiance (SID) from SARAH3, CM SAF – full extent (-65, -65, 65, 65)
Global horizontal irradiance (SIS) from SARAH3, CM SAF – full extent (-65, -65, 65, 65)
Air temperature at 2 meters (T2M) from ERA5-Land, ECMWF – SARAH3 extent (clipped using the original data from EOS)
Wind speed at 2 meters (WS2M) from ERA5-Land, ECMWF – SARAH3 extent (clipped using the original data from EOS)
Spectral effect (both cSi and CdTe) using PVGIS 5.3 data – SARAH3 extent (clipped locally). A conversion script (*.bin ▶️ *.tif) was used, followed by storing the data in a NetCDF file with 12 timestamps for 2013. The data were then processed by the script available here.
Convert time offset data (SARAH3, CM SAF) from *.nc to *.zarr with spatial chunking.
Horizon profiles still need to be processed. The approach is to reuse PVGIS 5.3 data, following a similar procedure to the spectral effect.
Additional meteorological variables must be processed in the future for the TMY endpoint.

Software for data processing

As mentioned above, the scripts from pvgis-time-series were used for data processing.
Inside the /software/scripts partition, the following can be found:

The Python virtual environment used for processing (.process_data)
The processing scripts

Data location on the server

All data are stored in the /data partition of the server.

SIS and SID data:
- Raw data in /data/sarah3/sis/... and /data/sarah3/sid/...
- Processed results in /data/sarah3/results/, with file naming convention:
  VARIABLE_NAMEin_chunks_SPATIALCHUNKSIZE_SPATIALCHUNKSIZE.zarr
T2M and WS2M data:
- Raw data in /data/era5land/sarah3_extent/...
- Processed results in /data/era5land/results/, with file naming convention:
  cds_era5_land_VARIABLENAME_chunks_SPATIALCHUNKSIZE_SPATIALCHUNKSIZE.zarr
Spectral effect data:
- Raw data in /data/spectral/raw/...
- Processed results in /data/spectral/results/, with file naming convention:
  spectral_effect_PANELTECHNOLOGY_chunks_SPATIALCHUNKSIZE_SPATIALCHUNKSIZE.zarr

Software state, branch, and issues to be addressed

The active branch on the production server is 338-gitlab-production-server-version

Issues

Update the spatial extent of the application to match the SARAH3 extent. A land/ocean mask may be needed, as no data are available over ocean areas.
Update default start_time and end_time values from 2013-01-01, 2013-12-31 to 2014-01-01, 2014-12-31 respectively. Also add minimum and maximum values.
Update the logic in the application to select the appropriate spectral effect file based on the user's selected panel technology.
Build and expose the documentation along with the app, and fix any broken links or other documentation issues.