Adapt loader to convert cumulative kWh to power (kW)

[PhilippSchmelter]

Closes #1

This pull request updates the data format and preprocessing logic to align with a new dataset structure. The changes include modifying the raw data file to include additional metadata and adapting the data loader to handle the new format while introducing a calculated power column.

Changes to data format:

• data/raw/dummy.csv: Replaced the old dataset structure with a new format that includes metadata fields (e.g., id, measurement_category, location_id) and updated time series data columns from ts and value to Zeitstempel and Messwert.

Updates to data preprocessing:

• src/preprocessing/loader.py:
  • Adjusted the load_timeseries function to skip metadata rows (skiprows=21) and use the updated column names (Zeitstempel and Messwert) for parsing.
  • Renamed columns to timestamp and cum_kwh, then calculated a new power column as the difference of cum_kwh multiplied by 4 (to convert cumulative kWh to power in 15-minute intervals).
  • Dropped rows with missing power values and removed the intermediate cum_kwh column for a cleaner dataset.

[Copilot]

Pull Request Overview

This pull request adapts the data loader to handle a new dataset structure by updating CSV parsing parameters and computing a power column from the cumulative kWh data.

• Updated CSV file reading to skip metadata rows and use new column names.
• Renamed columns to reflect the new data format and computed a power column by differencing the cumulative kWh values.
• Dropped intermediary columns to deliver a cleaner dataset.

Files not reviewed (1)

• data/raw/dummy.csv: Language not supported