Forecast

Forecast analysis

Forecast analysis is a pivotal domain within Data Science. It leverages machine learning techniques to determine the maximum likelihood of an event. Predictive analysis within this module extrapolates and anticipates future values by harnessing historical data sourced both internally and externally across the enterprise.

This analytics segment delves into historical datasets, extracting valuable trends and patterns through statistical analysis. It formulates strategies based on forecasting models to process new data and effectively predict the most optimal outcomes.

Consider scenarios like forecasting the volume of calls anticipated for the upcoming month or quarter in a call center or predicting the average handling duration for call agents supervised by XYZ. Before initiating the configuration for predictive analysis, it’s essential to contextualize the specific Key Performance Indicator (KPI) and seamlessly complete the remaining steps in the process.

Steps to perform forecast analysis in Lumenore

Step 1: After accessing “Do You Know,” select “Forecast.”

Step 2: Choose a Schema and click “Next.”

Note: Ensure prerequisites (A KPI, Date, and Attribute) for forecast analysis are met.

Step 3: Select the following to configure:

• Select forecast metric: Select a singular metric to conduct predictive analysis.
• Select input measure: Choose the input metric for predicting the output metric.
• Select the forecast date attribute: To structure and summarize the data based on a date attribute like ship date or order date, select a frequency such as yearly, quarterly, monthly, weekly, etc.

Algorithm: You have two options: Auto-select and Manual. In Auto-select, Lumenore will choose the best machine learning algorithm for the given data. In the Manual, you can choose the algorithm you want to use for forecasting based on your preferences.

Below are the algorithms for the forecast:

You can manually select them: Various time series forecasting algorithms are available, each with distinct strengths.

VARIMA:

• VARIMA models are an extension of ARIMA (Auto Regressive Integrated Moving Average) models.
• Deals with multiple time series that may correlate or influence each other.

Polynomial Regression:

• Regression analysis where the relationship between the independent variable (or predictor) and the dependent variable (or response) is modelled as an nth-degree polynomial.

Regression:

• Used for predicting future values based on past observations.
• In Lumenore, the optimal window size is automatically selected, simplifying the forecasting process.

Exponential Smoothing:

• Emphasizes recent observations for short-term forecasts.
• Efficient and applicable to a wide range of time series data.

SARIMA (Seasonal-ARIMA):

• Models’ seasonality in the series.
• ARIMA (Auto-Regressive Integrated Moving Average) with seasonal parameters.

Theta Method:

• Extracts linear trend and curvature.
• Simple yet effective, especially for time series practitioners.

Prophet:

• Fits non-linear trends with yearly, weekly, and daily seasonality.
• Suitable for time series with strong seasonal effects.
• Select forecast time range: Duration into the future for which forecasts are generated based on historical data and predictive models. (Year range 1 to 10)
• Selecting Split Ratio (Train: Test): This determines the proportion of data used for training and testing the model. Commonly used ratios include 70:30 or 80:20, which prevent overfitting and ensure the model generalizes well.
• Remove outliers: Remove outliers from the selected KPIs.
• Do you want to add filters??Optionally add filters.
• Advance setting: Users can perform predictive analytics by partitioning datasets based on any chosen attribute. This allows for comprehensive insights into key performance indicators (KPIs) across various segments, helping to uncover valuable patterns and trends. The maximum limit for top, bottom and custom selections is 10.

Note: If you wish to apply a filter, a window for creating filters will appear. As shown in the Trend Analysis, establish filters by groups or conditions as needed.

Step 4: Customize the insights narration and click “Save.”

Step 5: Name the insight for future reference and click “Save/Update.”

Step 6: A new window appears; click “Execute Now” to generate insights.

Upon initiation of execution, the system will undergo four background processes. You can also terminate the execution at any point before its completion.

For clarity, an insight for forecast analysis has been created using profit (forecast metric), sales & quantity (input measure), order date-quarterly (forecast date attribute), and Train 80% Test 20% (Split ratio).

The resulting forecast for overall profit and the top 4 countries based on profit is shown in the chart, with the dash line curve representing the predicted values.

Note: To learn about SMAPE and MSE, click on the “SMAPE” and “MSE” links on the insight screen.

SMAPE (Symmetric Mean Absolute Percentage Error) and MSE (Mean Squared Error) are two common metrics used to evaluate the accuracy of forecasting models. Here’s a breakdown of these metrics and how you can use them to assess your forecasting accuracy:

1. SMAPE (Symmetric Mean Absolute Percentage Error):

• Definition: SMAPE measures the percentage error between the forecasted and actual values, scaled by their average. It is “symmetric” because it gives equal weight to overestimation and underestimation.
• Formula:

• Range:
  • 0% (perfect accuracy) to 200% (worst possible error).
• Interpretation:
  • Low SMAPE (closer to 0) indicates better forecasting accuracy.
  • Values below 20% are generally considered acceptable, but this depends on the context.

2. MSE (Mean Squared Error):

• Definition: MSE measures the average of the squared differences between actual and forecasted values. Squaring the errors emphasizes larger errors more than smaller ones.
• Formula:

• Range:
  • Non-negative; the lower the MSE, the better the forecast.
• Interpretation:
  • A low MSE indicates high accuracy.
  • It is sensitive to outliers; even one large error can significantly increase MSE.

Using SMAPE and MSE to Evaluate Forecast Accuracy:

1. Compare Across Models:
   • Use SMAPE and MSE to compare the performance of different forecasting models. The model with the lowest SMAPE and MSE is usually the most accurate.
2. Thresholds:
   • SMAPE: Less than 10-20% is good for many industries, but the acceptable range depends on the domain (e.g., weather forecasting might tolerate higher errors).
   • MSE: Look at the scale of the data. For example, if your actual values are typically in the range of 1000, an MSE of 1 might be excellent, while for values around 1, it might be terrible.
3. Context Matters:
   • High SMAPE or MSE could still be acceptable in highly volatile domains where errors are unavoidable.
   • Always normalize these metrics to your dataset’s characteristics to understand their practical significance.

Good Practices for Forecasting Evaluation:

• Combine SMAPE and MSE with other metrics (e.g., MAE, RMSE) to get a well-rounded view of model performance.
• Visualize the actual vs. predicted values to detect patterns or systematic errors.
• Use a validation dataset to check if the model generalizes well.