generated from openclimatefix/ocf-template
-
-
Notifications
You must be signed in to change notification settings - Fork 34
Open
Labels
Description
We have moved to primarily use xarray-tensorstore as a backend, but in some places it is possible for dask to creep back in. I recently had this where there was an extra unused non-dimensional coordinate in our ECMWF data. This coordinate wa maintained up to the compute method and since it uses dask, even though the coordinate is small and has very few chunks, it slowed down my sampler by around 20-30%.
This is something that could keep coming up.
We should perhaps:
- Remove non-dimensional coordinates in the load functions?
- Add a warning to the user if any datasets are dask arrays