How to resample time series data in Python

Time series data is a sequence of observations collected at fixed time intervals. The data can come from any field, such as finance, economics, health and environmental sciences. The time series data we collect may sometimes have different frequencies or resolutions, which may not be suitable for our analysis and data modeling processes. In this case, we can resample the time series data by upsampling or downsampling, thereby changing the frequency or resolution of the time series. This article will introduce different methods to upsample or downsample time series data.

Upsampling

Upsampling means increasing the frequency of the time series data. This is usually done when we need a higher resolution or more frequent observations. Python provides several methods for upsampling time series data, including linear interpolation, nearest neighbor interpolation, and polynomial interpolation.

Syntax

DataFrame.resample(rule, *args, **kwargs)
DataFrame.asfreq(freq, method=None)
DataFrame.interpolate(method='linear', axis=0, limit=None, inplace=False, limit_direction='forward', limit_area=None)

it's here,

• The resample function is a method provided by the pandas library to resample time series data. It is applied on a DataFrame and takes the rule parameter, which specifies the desired frequency for resampling. Additional arguments (*args) and keyword arguments (**kwargs) can be provided to customize the resampling behavior, such as specifying the aggregation method or handling missing values.

• The asfreq method is used in conjunction with the resample function to convert the frequency of the time series data. It takes the freq parameter, which specifies the desired frequency string for the output. The optional method parameter allows specifying how to handle any missing values ​​introduced during the resampling process, such as forward filling, backward filling, or interpolation.

• Interpolation method is used to fill missing values ​​or gaps in time series data. It interpolates according to the specified method (e.g. 'linear', 'nearest', 'spline') to estimate values ​​between existing observations. Additional parameters can control the axis of interpolation, the padding limit for consecutive NaN values, and whether to modify the DataFrame in place or return a new DataFrame.

Linear interpolation

Linear interpolation is used for upsampling time series data. It fills gaps by drawing straight lines between data points. Linear interpolation can be implemented using the resample function in the pandas library.

Example

Example

In the below example, we have a time series DataFrame with three observations on non-consecutive dates. We convert the 'Date' column to a datetime format and set it as the index. The resample function is used to upsample the data to a daily frequency ('D') using the asfreq method. Finally, the interpolate method with the 'linear' option fills the gaps between the data points using linear interpolation. The DataFrame, df_upsampled, contains the upsampled time series data with interpolated values .

import pandas as pd

# Create a sample time series DataFrame
data = {'Date': ['2023-06-01', '2023-06-03', '2023-06-06'],
        'Value': [10, 20, 30]}
df = pd.DataFrame(data)
df['Date'] = pd.to_datetime(df['Date'])
df.set_index('Date', inplace=True)

# Upsample the data using linear interpolation
df_upsampled = df.resample('D').asfreq().interpolate(method='linear')

# Print the upsampled DataFrame
print(df_upsampled)

Output

                Value
Date                 
2023-06-01  10.000000
2023-06-02  15.000000
2023-06-03  20.000000
2023-06-04  23.333333
2023-06-05  26.666667
2023-06-06  30.000000

Nearest neighbor interpolation

Nearest neighbor interpolation is a simple method that fills the gaps between data points with the nearest available observation. This method can be useful when the time series exhibits abrupt changes or when the order of observations matters. The interpolate method in pandas can be used with the 'nearest' option to perform nearest neighbor interpolation.

Example

Example

In the above example, we use the same original DataFrame as before. After resampling with the 'D' frequency, the interpolate method with the 'nearest' option fills the gaps by copying the nearest available observation. The resulting DataFrame, df_upsampled , now has a daily frequency with the nearest neighbor interpolation.

import pandas as pd

# Create a sample time series DataFrame
data = {'Date': ['2023-06-01', '2023-06-03', '2023-06-06'],
        'Value': [10, 20, 30]}
df = pd.DataFrame(data)
df['Date'] = pd.to_datetime(df['Date'])
df.set_index('Date', inplace=True)

# Upsample the data using nearest neighbor interpolation
df_upsampled = df.resample('D').asfreq().interpolate(method='nearest')

# Print the upsampled DataFrame
print(df_upsampled)

Output

            Value
Date             
2023-06-01   10.0
2023-06-02   10.0
2023-06-03   20.0
2023-06-04   20.0
2023-06-05   30.0
2023-06-06   30.0

Downsampling

Downsampling is used to reduce the frequency of time series data, typically to obtain a broader view of the data or to simplify analysis. Python offers different downsampling techniques, such as averaging, summing, or maximizing values ​​over a specified time interval.

Syntax

DataFrame.mean(axis=None, skipna=None, level=None, numeric_only=None, **kwargs)

Here, an aggregation method, such as mean, sum, or maximum, is applied after resampling to compute a single value representing the grouped observations within each resampling interval. These methods are typically used when downsampling data. They can be applied directly to a resampled DataFrame, or they can be used in conjunction with a resampling function to aggregate data based on a specific frequency (such as weekly or monthly) by specifying appropriate rules.

Mean Downsampling

mean downsampling

Mean downsampling calculates the average of the data points within each interval. This method is useful when processing high-frequency data and obtaining representative values ​​for each interval. You can use the resample function in conjunction with the mean method to perform mean downsampling.

Example

示例

In the below example, we start with a daily time series DataFrame spanning the entire month of June 2023. The resample function with the 'W' frequency downsamples the data to weekly intervals. By applying the mean method, we obtain the average value within each week. The resulting DataFrame, df_downsampled, contains the mean-downsampled time series data.

import pandas as pd

# Create a sample time series DataFrame with daily frequency
data = {'Date': pd.date_range(start='2023-06-01', end='2023-06-30', freq='D'),
        'Value': range(30)}
df = pd.DataFrame(data)
df.set_index('Date', inplace=True)

# Downsampling using mean
df_downsampled = df.resample('W').mean()

# Print the downsampled DataFrame
print(df_downsampled)

输出

            Value
Date             
2023-06-04    1.5
2023-06-11    7.0
2023-06-18   14.0
2023-06-25   21.0
2023-07-02   27.0

Maximum Downsampling

最大降采样计算并设置每个间隔内的最高值。此方法适用于识别时间序列中的峰值或极端事件。在前面的示例中使用max而不是mean或sum允许我们执行最大降采样。

Example

示例

In the below example, we start with a daily time series DataFrame spanning the entire month of June 2023. The resample function with the 'W' frequency downsamples the data to weekly intervals. By applying the max method, we obtain the Maximum value within each week. The resulting DataFrame, df_downsampled, contains the maximum-downsampled time series data.

import pandas as pd
# Create a sample time series DataFrame with daily frequency
data = {'Date': pd.date_range(start='2023-06-01', end='2023-06-30', freq='D'),
        'Value': range(30)}
df = pd.DataFrame(data)
df.set_index('Date', inplace=True)

# Downsampling using mean
df_downsampled = df.resample('W').max()

# Print the downsampled DataFrame
print(df_downsampled)

输出

            Value
Date             
2023-06-04      3
2023-06-11     10
2023-06-18     17
2023-06-25     24
2023-07-02     29

结论

在本文中，我们讨论了如何使用Python对时间序列数据进行重新采样。Python提供了各种上采样和下采样技术。我们探讨了线性和最近邻插值用于上采样，以及均值和最大值插值用于下采样。您可以根据手头的问题使用任何一种上采样或下采样技术。

The above is the detailed content of How to resample time series data in Python. For more information, please follow other related articles on the PHP Chinese website!