How to Use Accurate Forecasting Techniques

Modeling something like time series goes past just throwing features in a model. In the world of time series data, each observation is associated with a specific time point, and part of our goal is to harness the power of temporal dependencies. Enter autoregression and lagging -  concepts that taps into the correlation between current and past observations to make forecasts.  At its core, autoregression involves modeling a time series as a function of its previous values. The current value relies on its historical counterparts. To dive a bit deeper, we use lagged values as features to predict the next data point. For instance, in a simple autoregressive model of order 1 (AR(1)), we predict the current value based on the previous value multiplied by a coefficient. The coefficient determines the impact of the past value on the present one only one time period previous. One popular approach that can be used in conjunction with autoregression is the ARIMA (AutoRegressive Integrated Moving Average) model. ARIMA is a powerful time series forecasting method that incorporates autoregression, differencing, and moving average components. It's particularly effective for data with trends and seasonality. ARIMA can be fine-tuned with parameters like the order of autoregression, differencing, and moving average to achieve accurate predictions. When I was building ARIMAs for econometric time series forecasting, in addition to autoregression where you're lagging the whole model, I was also taught to lag the individual economic variables. If I was building a model for energy consumption of residential homes, the number of housing permits each month would be a relevant variable. Although, if there’s a ton of housing permits given in January, you won’t see the actual effect of that until later when the houses are built and people are actually consuming energy! That variable needed to be lagged by several months. Another innovative strategy to enhance time series forecasting is the use of neural networks, particularly Recurrent Neural Networks (RNNs) or Long Short-Term Memory (LSTM) networks. RNNs and LSTMs are designed to handle sequential data like time series. They can learn complex patterns and long-term dependencies within the data, making them powerful tools for autoregressive forecasting. Neural networks are fed with past time steps as inputs to predict future values effectively. In addition to autoregression in neural networks, I also used lagging there too! When I built an hourly model to forecast electric energy consumption, I actually built 24 individual models, one for each hour, and each hour lagged on the previous one. The energy consumption and weather of the previous hour was very important in predicting what would happen in the next forecasting period. (this model was actually used for determining where they should shift electricity during peak load times). Happy forecasting!

A poor demand forecast destroys profits and cash. This infographic shows 7 forecasting techniques, pros, cons, & when to use: 1️⃣ Moving Average ↳ Averages historical demand over a specified period to smooth out trends ↳ Pros: simple to calculate and understand  ↳ Cons: lag effect; may not respond well to rapid changes ↳ When: short-term forecasting where trends are relatively stable 2️⃣ Exponential Smoothing ↳ Weights recent demand more heavily than older data ↳ Pros: responds faster to recent changes; easy to implement ↳ Cons: requires selection of a smoothing constant ↳ When: when recent data is more relevant than older data 3️⃣ Triple Exponential Smoothing  ↳ Adds components for trend & seasonality ↳ Pros: handles data with both trend and seasonal patterns ↳ Cons: requires careful parameter tuning ↳ When: when data has both trend and seasonal variations 4️⃣ Linear Regression ↳ Models the relationship between dependent and independent variables ↳ Pros: provides a clear mathematical relationship ↳ Cons: assumes a linear relationship ↳ When: when the relationship between variables is linear 5️⃣ ARIMA ↳ Combines autoregression, differencing, and moving averages ↳ Pros: versatile; handles a variety of time series data patterns ↳ Cons: complex; requires parameter tuning and expertise ↳ When: when data exhibits autocorrelation and non-stationarity 6️⃣ Delphi Method ↳ Expert consensus is gathered and refined through multiple rounds ↳ Pros: leverages expert knowledge; useful for long-term forecasting ↳ Cons: time-consuming; subjective and may introduce bias ↳ When: historical data is limited or unavailable, low predictability 7️⃣ Neural Networks ↳ Uses AI to model complex relationships in data ↳ Pros: can capture nonlinear relationships; adaptive and flexible ↳ Cons: requires large data sets; can be a "black box" with less interpretability ↳ When: for complex, non-linear data patterns and large data sets Any others to add?

Forecast accuracy starts with structure. We’re using neural networks. That’s the base. But we’ve also built a whole framework around it. We have multiple model structures that reflect different forecast objectives (whether it’s forecasting shipments, retail sales, or pricing trends). Each one is optimized based on what the customer is trying to plan. We also support scenario-based planning. So if a client wants to run a “what if” model:  For instance, what happens if we drop price by 10%, or what happens if we start the promo one month earlier? We can plug that in and reforecast with those new inputs. We’ve seen strong accuracy from the start. MAPE has consistently been under 2%, which we’re pretty happy with. We also use persistent weighting, so the model updates based on incoming data and changes in inputs over time. And the weighting holds unless there’s a clear reason to shift it. That combination:  ➡️ Neural network base  ➡️ Scenario structure  ➡️ Persistent weighting Is what lets us give teams something they can actually use in planning. It’s something that reflects how they think.