9. API

9.1. High level data reading

pyrnet.pyrnet.get_elements(url, tag_name='dataset', attribute_name='urlPath')

Get elements from an XML file

pyrnet.pyrnet.get_pyrnet_mapping(fn: str, date)

Parse box - serial number mapping json file

Parameters:

  • fn (str) – Path of the mapping.json

  • date (list, ndarray, or scalar of type float, datetime or datetime64) – A representation of time. If float, interpreted as Julian date.

Returns:

Calibration dictionary sorted by box number.

Return type:

dict

pyrnet.pyrnet.lookup_fnames(date, *, station, lvl, campaign, collection)

Parse Thredds server files and return list of filenames matching the date, station, campaign and collection configuration.

pyrnet.pyrnet.parse_thredds_catalog(url, fname_format)

Parse Thredds server catalog and return pd.Dataframe of file name format variables.

pyrnet.pyrnet.read_calibration(cfile: str, cdate)

Parse calibration json file

Parameters:

  • cfile (str) – Path of the calibration.json

  • cdate (list, ndarray, or scalar of type float, datetime or datetime64) – A representation of time. If float, interpreted as Julian date.

Returns:

Calibration dictionary sorted by box number.

Return type:

dict

pyrnet.pyrnet.read_hdcp2(dt, fill_gaps=True, campaign='hope_juelich')

Read HDCP2-formatted datafiles from the pyranometer network

Parameters:

  • dt (datetime.date) – The date of the data to read

  • fill_gaps (bool) – A flag indicating whether gaps should be filled by interpolation

  • campaign (str) – specify campaign [‘eifel’,’hope_juelich’,’hope_melpitz’,’lindenberg’,’melcol’]

Returns:

dataset – The pyranometer network observations

Return type:

xarray.Dataset

pyrnet.pyrnet.read_pyrnet(date, campaign)

Read pyrnet data and add coordinates

pyrnet.pyrnet.read_thredds(dates, *, campaign, stations=None, lvl='l1b', collection=None, freq='1s', drop_vars=None)

Read PyrNet data (processed with pyrnet package) from the TROPOS thredds server. Returns one xarray Dataset merged to match the dates and stations input. :param dates: A representation of time. If float, interpreted as Julian date. :type dates: list, ndarray, or scalar of type float, datetime or datetime64 :param campaign: Campaign identifier. :type campaign: str :param stations: PyrNet station numbers. If None, read all stations available. :type stations: list, ndarray, or scalar of type int or None :param lvl: Data processing level -> ‘l1a’, ‘l1b’. The default is ‘l1b’. :type lvl: str :param collection: Collection number. If None, the latest available collection is looked up. The default is None. :type collection: int or None :param freq: Pandas date frequencey description string. The default is ‘1s’. :type freq: str :param drop_vars: List of variables to drop from datasets to speed up merging process. :type drop_vars: list of string or None

Returns:

Merged Dataset including all dates and stations specified by the input.

Return type:

xarray.Dataset

9.2. Utilities

pyrnet.utils.calc_apparent_coszen(pitch, yaw, zen, azi)

Calculate cosine of apparent zenith angle

9. Parameters:

pitch: float, array of float

Platform pitch angle (degrees) - Angle between vertical and platfrom normal vector, e.g. platform zenith angle

yaw: float, array of float

Platform yaw angle (degrees) - Angle positive clockwise from north, e.g. platfrom azimuth angle

zen: flaot, array of float

Solar zenith angle (degrees)

azi: float, array of float

Solar azimuth angle (degrees)

9. Returns:

coszen: float, array of float

The cosine of the platform apparent zenith angle (angle between solar and platform normal vector)

pyrnet.utils.check_tilted(x)

Check x attrs ‘hangle’ and ‘vangle’ if available and >0.1 degree.

Parameters:

x (xr.DataArray)

Returns:

True if vangle attribute > 0.1 degree. Same length as ‘x.vangle’.

Return type:

list of bools

pyrnet.utils.gauss_fwin(J: float, N: int = 86400) → ndarray[tuple[int, ...], dtype[_ScalarType_co]]

Convert scale parameter to FWHM of Normal distribution see [https://en.wikipedia.org/wiki/Full_width_at_half_maximum#Normal_distribution]

Parameters:

  • J (float) – scale parameter for FWHM, with FWHM=60*2**J

  • N (int) – Number of points in the output window. If zero, an empty array is returned. An exception is thrown when it is negative. The default is 86400 (seconds per day).

Returns:

Frequency response of the gaussian window

Return type:

ndarray

pyrnet.utils.gauss_fwin_fwhm(fwhm: float, N: int = 86400) → ndarray[tuple[int, ...], dtype[_ScalarType_co]]

Convert scale parameter to FWHM of Normal distribution see [https://en.wikipedia.org/wiki/Full_width_at_half_maximum#Normal_distribution]

Parameters:

  • fwhm (float) – FWHM of the gaussian distribution

  • N (int) – Number of points in the output window. If zero, an empty array is returned. An exception is thrown when it is negative. The default is 86400 (seconds per day).

Returns:

Frequency response of the gaussian window

Return type:

ndarray

pyrnet.utils.get_attrs_enc(d: dict) -> (<class 'dict'>, <class 'dict'>)

Split variable attributes in attributes and encoding-attributes.

pyrnet.utils.get_var_attrs(d: dict) → dict

Parse cf-compliance dictionary.

Parameters:

d (dict) – Dict parsed from cf-meta json.

Returns:

Dict with netcdf attributes for each variable.

Return type:

dict

pyrnet.utils.get_xy_coords(lon, lat, lonc=None, latc=None)

Calculate Cartesian coordinates of network stations, relative to the mean lon/lat of the stations

pyrnet.utils.make_iter(x)

Check if x is an iterable, if not make it so and return np.array(x).

pyrnet.utils.omit(blacklist: list[str], d: dict) → dict

Omit blacklisted keys from input dict.

pyrnet.utils.pairwise_distance_matrix(x: _Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], y: _Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes]) → ndarray[tuple[int, ...], dtype[_ScalarType_co]]

Get square matrix with Euclidian distances of stations

Parameters:

  • x (array_like) – X coordinates

  • y (array_like) – Y coordinates

Returns:

A square matrix with Euclidian distances of stations

Return type:

ndarray

pyrnet.utils.pick(whitelist: list[str], d: dict) → dict

Keep only whitelisted keys from input dict.

pyrnet.utils.read_json(fpath: str, *, object_hook: type = <class 'addict.addict.Dict'>, cls=None) → dict

Parse json file to python dict.

pyrnet.utils.smooth(y: _Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], J: float, axis: int = 0) → ndarray[tuple[int, ...], dtype[_ScalarType_co]]

Smooth data with gaussian window by convolution

Parameters:

  • y (array_like) – Input array.

  • J (float) – Scale parameter for the FWHM (FWHM=60*2**J) of the gaussian window to be convolved with the input array.

  • axis (int, optional) – Axis over which to smoothing is applied.

Returns:

Smoothed array of the same shape as the input array y.

Return type:

ndarray

pyrnet.utils.smooth_fwhm(y: _Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], fwhm: float, axis: int = 0) → ndarray[tuple[int, ...], dtype[_ScalarType_co]]

Smooth data with gaussian window by convolution

Parameters:

  • y (array_like) – Input array.

  • fwhm (float) – FWHM of the gaussian window to be convolved with the input array.

  • axis (int, optional) – Axis over which to smoothing is applied.

Returns:

Smoothed array of the same shape as the input array y.

Return type:

ndarray

pyrnet.utils.to_datetime64(time, epoch=np.datetime64('2000-01-01T12:00'))

Convert various representations of time to datetime64.

Parameters:

  • time (list, ndarray, or scalar of type float, datetime or datetime64) – A representation of time. If float, interpreted as Julian date.

  • epoch (np.datetime64, default JD2000.0) – The epoch to use for the calculation

Return type:

datetime64 or ndarray of datetime64

9.3. Processing

pyrnet.logger.adc_binning(rec_adc, time, bins=86400)

Binning and averaging of ADC samples

Parameters:

  • rec_adc (ndarray) – The ADC records parsed from the logger.

  • time (ndarray of time objects) – Sample time of ADC records.

  • bins (int) – Number of desired bins per day. The default is 86400, which result in mean values of 1 second steps per day. Maximum resolution is 86400000.

Returns:

Binned ADC records and corresponding time.

Return type:

ndarray, ndarray(datetime64)

pyrnet.logger.get_adc_time(rec_adc)

Get Milliseconds from Start of ADC measurement.

Parameters:

rec_adc (ndarray) – The digital counts of the ADC from the logger file.

Returns:

Milliseconds from start of the measurement.

Return type:

ndarray

pyrnet.logger.interpolate_coords(rec_gprmc, time)

Interpolate lat and lon from gps records

Parameters:

  • rec_gprmc (recarray) – The GPRMC records from the logger file

  • time (list or array of time objects)

Returns:

lat and lon interpolated from GPS records to time

Return type:

ndarray, ndarray

pyrnet.logger.parse_adc(s)

Parse an ADC record

Parameters:

s (string) – The ADC record

Returns:

t – A tuple of digital counts of the ADC

Return type:

tuple

pyrnet.logger.parse_gprmc(s, date_of_measure=np.datetime64('2026-05-18T09:42:39'))

Parse a string with a GPRMC GPS record

Parameters:

  • s (string) – A GPRMC record

  • date_of_measure (datetime or datetime64) – A rough time, when the measurements happen to account for GPS rollover. Precise datetime is only necessary for the period of 2019-05-06 to 2019-08-17. Otherwise, providing a year is sufficient. The default is np.datetime64(“now”), which is sufficient for all measurements conducted later than 2019-08-17.

Returns:

gprmc – A tuple with datetime64,status,lat,lon

Return type:

tuple

pyrnet.logger.read_records(fname: str, date_of_measure: datetime64 = np.datetime64('2026-05-18T09:42:39'))

Read the GPRMC and ADC records from the pyranometer logger files

Parameters:

  • fname (string) – The filename of the logger file

  • date_of_measure (numpy.datetime64) – Date of measurement to account for gps rollover

Returns:

  • rec_adc (ndarray) – The 10bit ADC readings

  • rec_gprmc (recarray) – The GPRMC GPS records

pyrnet.logger.sync_adc_time(adctime, gpstime, iadc, check_results=True)

Synchronize the ADC time to the GPS records

Parameters:

  • adctime (ndarray) – Milliseconds from start of the ADC measurement.

  • gpstime (ndarray) – GPS time

  • iadc (ndarray of int) – Index of the last ADC sample before a GPS record has been stored.

  • check_results (bool) – If True, check plausibility of fitted slope and offset (abs(slope)<10s/day; abs(offset)<2s). The default is True

Returns:

time – The time of the ADC records

Return type:

ndarray(datetime64[ms])

pyrnet.reports.get_qcflag(qc_clean, qc_level)

Aggregate quality flags.

Parameters:

  • qc_clean (int) – [0,1,2,3] [clean, slight-, moderate-, strong covered]

  • qc_level (int) – [0,1,2] [good, slight misalignment, strong misalignment]

Returns:

aggregated quality flagg [0-11]

Return type:

int

pyrnet.reports.get_responses(*, fn: str | PathLike[str] | ReadCsvBuffer[bytes] | ReadCsvBuffer[str] | None = None, online: dict | None = None) → DataFrame

Get LimeSurvey responses as pandas Dataframe providing a file or online download information.

Parameters:

  • fn (str, path object or file-like object) – Any pandas readable representation of the LimeSurvey response export file (.csv, sep=;, answer and question codes).

  • online (dict) –

    Dictionary of information required to download the responses via limepy:

    • base_url -> limesurvey remote_control url

    • user_name -> account name

    • password

    • user_id -> ID of account user (usually 1)

    • sid -> Survey ID

    Minimal information stored in online is the base_url, other information will then be filled via user input promt.

Returns:

parsed responses csv file

Return type:

pd.Dataframe

pyrnet.reports.parse_report(df: DataFrame, date_of_maintenance: float | datetime | datetime64 | None, stations: list | int | None = None) → dict

Use pandas.read_csv (sep=;) to parse the survey report.

Parameters:

  • df (Dataframe) – LimeSurvey response parsed as pandas Dataframe.

  • date_of_maintenance (float, datetime, datetime64 or None) – A rough date of maintenance (at least day resolution). If float, interpreted as Julian day from 2000-01-01T12:00. If None, the most recent logbook entries will be parsed.

  • stations (list, int or None) – Selection of station (box) numbers to parse the report for. If None, parse all available stations. The default is None.

Returns:

Dictionary storing maintenance flags and notes by PyrNet box number.

Return type:

dict

pyrnet.reports.read_logbook(lfile)

Load logbook file and store it as dictionary of rec arrays with stID keys.

Parameters:

cfile (string) – path and filename of loogbook file -> should be .xls file each sheet represent a maintenance periode. First row of .xls file have to include column names box, date, clean, clean_tilt, level, level_tilt, Hangle, Vangle, notes

Returns:

logbook – dict of recarray for each station ID including quality flags from each maintenance cicle.

Return type:

dict