Skip to content

Extended the description of quantile mapping with text and images #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@

This Python module serves as a collection of different scale- and distribution-based bias correction techniques for climatic research

The documentation is available at: [https://python-kraken-sdk.readthedocs.io/en/stable/](https://python-kraken-sdk.readthedocs.io/en/stable/)
The documentation is available at: [https://python-cmethods.readthedocs.io/en/stable/](https://python-cmethods.readthedocs.io/en/stable/)

> ⚠️ For the application of bias corrections on _lage data sets_ it is recomanded to use the command-line tool [BiasAdjustCXX](https://github.com/btschwertfeger/BiasAdjustCXX) since bias corrections are complex statistical transformation which are very slow in Python compared to the C++ implementation.

Expand Down Expand Up @@ -78,6 +78,17 @@ All methods except the `adjust_3d` function requires that the input data sets on
| `quantile_delta_mapping` | Quantile Delta Mapping (additive and multiplicative) |
| `adjust_3d` | requires a method name and the respective parameters to adjust all time series of a 3-dimensional data set |

Except for the variance scaling, all methods can be applied on stochastic and non-stochastic
climate variables. Variance scaling can only be applied on non-stochastic climate variables.

- Stochastic climate variables are those that are subject to random fluctuations
and are not predictable. They have no predictable trend or pattern. Examples of
stochastic climate variables include precipitation, air temperature, and humidity.

- Non-stochastic climate variables, on the other hand, have clear trend and pattern histories
and can be readily predicted. They are often referred to as climate elements and include
variables such as water temperature and air pressure.

---

<a name="installation"></a>
Expand Down Expand Up @@ -119,7 +130,7 @@ qdm_result = cm.adjust_3d( # 3d = 2 spatial and 1 time dimension
)
# to calculate the relative rather than the absolute change,
# '*' can be used instead of '+' (this is prefered when adjusting
# ratio based variables like precipitation)
# stochastic variables like precipitation)
```

Notes:
Expand Down
75 changes: 56 additions & 19 deletions cmethods/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,16 @@ class CMethods:
* Quantile Mapping :func:`cmethods.CMethods.quantile_mapping`
* Quantile Delta Mapping :func:`cmethods.CMethods.quantile_delta_mapping`

Except for the Variance Scaling all methods can be applied on both, interval- and ratio- based
variables. The Variance Scaling can only be applied on interval-based variables.
Except for the Variance Scaling all methods can be applied on both, stochastic and non-stochastic
variables. The Variance Scaling can only be applied on stochastic climate variables.

Stochastic climate variables are those that are subject to random fluctuations
and are not predictable. They have no predictable trend or pattern. Examples of
stochastic climate variables include precipitation, air temperature, and humidity.

Non-stochastic climate variables, on the other hand, have clear trend and pattern histories
and can be readily predicted. They are often referred to as climate elements and include
variables such as water temperature and air pressure.
"""

SCALING_METHODS = ["linear_scaling", "variance_scaling", "delta_method"]
Expand Down Expand Up @@ -362,8 +370,8 @@ def linear_scaling(
**kwargs,
) -> np.array:
r"""
The Linear Scaling bias correction technique can be applied on interval- and
ratio-based climate variables to minimize deviations in the mean values
The Linear Scaling bias correction technique can be applied on stochastic and
non-stochastic climate variables to minimize deviations in the mean values
between predicted and observed time-series of past and future time periods.

Since the multiplicative scaling can result in very high scaling factors,
Expand Down Expand Up @@ -412,8 +420,8 @@ def linear_scaling(
:type simp: xr.core.dataarray.DataArray
:param group: The grouping defines the basis of the mean, defaults to ``time.month``
:type group: Union[str, None], optional
:param kind: The kind of the correction, additive for interval- and multiplicative
for ratio-based variables, defaults to ``+``
:param kind: The kind of the correction, additive for non-stochastic and multiplicative
for stochastic climate variables, defaults to ``+``
:type kind: str, optional
:raises NotImplementedError: If the kind is not in (``+``, ``*``, ``add``, ``mult``)
:return: The bias-corrected time series
Expand Down Expand Up @@ -474,7 +482,7 @@ def variance_scaling(
**kwargs,
) -> np.array:
r"""
The Variance Scaling bias correction technique can be applied on interval-based
The Variance Scaling bias correction technique can be applied only on non-stochastic
climate variables to minimize deviations in the mean and variance
between predicted and observed time-series of past and future time periods.

Expand Down Expand Up @@ -532,7 +540,7 @@ def variance_scaling(
:type simp: xr.core.dataarray.DataArray
:param group: The grouping defines the basis of the mean, defaults to ``time.month``
:type group: Union[str, None], optional
:param kind: The kind of the correction, additive for interval-based variables,
:param kind: The kind of the correction, additive for non-stochastic climate variables
no other kind is available so far, defaults to ``+``
:type kind: str, optional
:raises NotImplementedError: If the kind is not in (``+``, ``add``)
Expand Down Expand Up @@ -600,8 +608,8 @@ def delta_method(
**kwargs,
) -> np.array:
r"""
The Delta Method bias correction technique can be applied on interval- and
ratio-based climate variables to minimize deviations in the mean values
The Delta Method bias correction technique can be applied on stochastic and
non-stochastic climate variables to minimize deviations in the mean values
between predicted and observed time-series of past and future time periods.

Since the multiplicative scaling can result in very high scaling factors,
Expand Down Expand Up @@ -651,8 +659,8 @@ def delta_method(
:type simp: xr.core.dataarray.DataArray
:param group: The grouping defines the basis of the mean, defaults to ``time.month``
:type group: Union[str, None], optional
:param kind: The kind of the correction, additive for interval- and multiplicative
for ratio-based variables, defaults to ``+``
:param kind: The kind of the correction, additive for non-stochastic and multiplicative
for stochastic climate variables, defaults to ``+``
:type kind: str, optional
:raises NotImplementedError: If the kind is not in (``+``, ``*``, ``add``, ``mult``)
:return: The bias-corrected time series
Expand Down Expand Up @@ -734,14 +742,39 @@ def quantile_mapping(
After the Quantile Mapping was applied, the mean is shifted back. Since it does not make sens
to take the whole mean to rescale the data, the month-dependent long-term mean is used.

In the following the equations of Alex J. Cannon (2015) are shown (without detrending):
In the following the equations of Alex J. Cannon (2015) are shown and explained (without detrending):

**Additive**:

.. math::

X^{*QM}_{sim,p}(i) = F^{-1}_{obs,h} \left\{F_{sim,h}\left[X_{sim,p}(i)\right]\right\}


The additive quantile mapping procedure consists of inserting the value to be
adjusted (:math:`X_{sim,p}(i)`) into the cumulative distribution function
of the modeled data of the control period (:math:`F_{sim,h}`). This determines,
in which quantile the value to be adjusted can be found in the modeled data of the control period
The following images show this by using :math:`T` for temperatures.

.. figure:: ../_static/images/qm-cdf-plot-1.png
:width: 600
:align: center
:alt: Determination of the quantile value

Fig 1: Inserting :math:`X_{sim,p}(i)` into :math:`F_{sim,h}` to determine the quantile value

This value, which of course lies between 0 and 1, is subsequently inserted
into the inverse cumulative distribution function of the reference data of the control period to
determine the bias-corrected value at time step :math:`i`.

.. figure:: ../_static/images/qm-cdf-plot-2.png
:width: 600
:align: center
:alt: Determination of the QM bias-corrected value

Fig 1: Inserting the quantile value into :math:`F^{-1}_{obs,h}` to determine the bias-corrected value for time step :math:`i`

**Multiplicative**:

.. math::
Expand All @@ -759,12 +792,16 @@ def quantile_mapping(
:type simp: xr.core.dataarray.DataArray
:param n_quantiles: Number of quantiles to respect/use
:type n_quantiles: int
:param kind: The kind of the correction, additive for interval- and multiplicative
for ratio-based variables, defaults to ``+``
:param kind: The kind of the correction, additive for non-stochastic and multiplicative
for stochastic climate variables, defaults to ``+``
:type kind: str, optional
:param detrended: If the extremes should be respected by applying month-dependent
detrending before and after applying the Quantile Mapping
:type detrended: bool, optional
:param val_min: Lower boundary for interpolation (only if ``kind="*"``, default: ``0``)
:type val_min: float, optional
:param val_max: Upper boundary for interpolation (only if ``kind="*"``, default: ``None``)
:type val_max: float, optional
:raises NotImplementedError: If the kind is not in (``+``, ``*``, ``add``, ``mult``)
:return: The bias-corrected time series
:rtype: np.array
Expand Down Expand Up @@ -990,8 +1027,8 @@ def quantile_delta_mapping(
:type simp: xr.core.dataarray.DataArray
:param n_quantiles: Number of quantiles to respect/use
:type n_quantiles: int
:param kind: The kind of the correction, additive for interval- and multiplicative
for ratio-based variables, defaults to ``+``
:param kind: The kind of the correction, additive for non-stochastic and multiplicative
for non-stochastic climate variables, defaults to ``+``
:type kind: str, optional
:raises NotImplementedError: If the kind is not in (``+``, ``*``, ``add``, ``mult``)
:return: The bias-corrected time series
Expand Down Expand Up @@ -1125,7 +1162,7 @@ def get_inverse_of_cdf(
xbins: Union[list, np.array],
) -> np.array:
r"""
Returns the inverse cumulative distribution function of with the following
Returns the inverse cumulative distribution function as:
:math:`F^{-1}_{x}\left[y\right]` where :math:`x` represents ``base_cdf`` and
``insert_cdf`` is represented by :math:`y`.

Expand All @@ -1152,7 +1189,7 @@ def get_adjusted_scaling_factor(

where:
- :math:`x` is ``factor``
- :math:`y` is ``max_scaling_factor``.
- :math:`y` is ``max_scaling_factor``

:param factor: The value to check for
:type factor: Union[int, float]
Expand Down
File renamed without changes
File renamed without changes
Binary file added docs/_static/images/qm-cdf-plot-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/images/qm-cdf-plot-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 13 additions & 2 deletions docs/src/introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ time periods. Data from past periods are used to adjust variables
from current and future time series so that their distributional
properties approximate possible actual values.

.. figure:: ../images/biasCdiagram.png
.. figure:: ../_static/images/biasCdiagram.png
:width: 600
:align: center
:alt: Schematic representation of a bias adjustment procedure
Expand All @@ -31,7 +31,7 @@ It is directly visible that the delta adjusted time series
(:math:`T^{*DM}_{sim,p}`) are much more similar to the observed data (:math:`T_{obs,p}`)
than the raw modeled data (:math:`T_{sim,p}`).

.. figure:: ../images/dm-doy-plot.png
.. figure:: ../_static/images/dm-doy-plot.png
:width: 600
:align: center
:alt: Temperature per day of year in modeled, observed and bias-adjusted climate data
Expand All @@ -56,6 +56,17 @@ All of these methods are intended to be applied on 1-dimensional time-series cli
This module also provides the function :func:`cmethods.CMethods.adjust_3d` that enables
the application of the desired bias correction method on 3-dimensinoal data sets.

Except for the variance scaling, all methods can be applied on stochastic and non-stochastic
climate variables. Variance scaling can only be applied on non-stochastic climate variables.

- Stochastic climate variables are those that are subject to random fluctuations
and are not predictable. They have no predictable trend or pattern. Examples of
stochastic climate variables include precipitation, air temperature, and humidity.

- Non-stochastic climate variables, on the other hand, have clear trend and pattern histories
and can be readily predicted. They are often referred to as climate elements and include
variables such as water temperature and air pressure.

Examples can be found in the `python-cmethods`_ repository and of course
within this documentation.

Expand Down