Interpolation And Extrapolation

When data is uploaded with missing dates or when performing predictions/simulations into the future, Alviss AI automatically handles missing values through interpolation (for gaps within the data) and extrapolation (for extending into the future or past). This ensures consistent and complete datasets for analysis, modeling, and visualization.

The process applies a sequence of filling methods to missing values (represented as NaN or null). Methods are applied in order until all missing values are resolved. The specific sequence depends on the variable type, as different data categories (e.g., sales, media spend) behave differently and require tailored handling to maintain accuracy and realism.

Key Methods Used

The following methods are employed to fill missing values:

• Interpolate: Performs linear interpolation between known values. This is ideal for filling gaps in the middle of a time series but does not handle leading (early) or trailing (late/future) missing values on its own.
• Forward Fill: Propagates the last known value forward to fill subsequent missing values. Useful for extending trends into the future (extrapolation) or handling trailing gaps.
• Backward Fill: Propagates the next known value backward to fill previous missing values. Useful for handling leading gaps or backfilling from future data points.
• Set Value: Sets all remaining missing values to a fixed value (default is 0). This is a fallback method to ensure no NaNs remain, often used for variables where assuming zero makes business sense (e.g., no activity recorded).

For interpolation (internal gaps), methods like Interpolate are prioritized where appropriate. For extrapolation (e.g., future predictions or simulations), Forward Fill or Set Value are commonly used to extend data logically. The system processes data along the time axis, assuming dates are in chronological order.

Rules by Variable Group

Each variable group has a predefined sequence of methods. These are applied in the listed order. If a method doesn't fully resolve the missing values, the next one is tried.

• Sales:

  • Set Value (to 0)

• Media Conversion Factor:

  • Interpolate
  • Backward Fill
  • Forward Fill
  • Set Value (to 0)

• Price:

  • Interpolate
  • Forward Fill
  • Backward Fill

• Profit:

  • Interpolate
  • Forward Fill
  • Backward Fill

• Cost:

  • Interpolate
  • Forward Fill
  • Backward Fill

• Competitor Price:

  • Interpolate
  • Forward Fill
  • Backward Fill

• Distribution:

  • Interpolate
  • Forward Fill
  • Backward Fill
  • Set Value (to 0)

• Competitor Distribution:

  • Interpolate
  • Forward Fill
  • Backward Fill
  • Set Value (to 0)

• Visits:

  • Interpolate
  • Forward Fill
  • Backward Fill
  • Set Value (to 0)

• Brand:

  • Interpolate
  • Forward Fill
  • Backward Fill
  • Set Value (to 0)

• Competitor Brand:

  • Interpolate
  • Forward Fill
  • Backward Fill
  • Set Value (to 0)

• Weather:

  • Interpolate
  • Forward Fill
  • Backward Fill
  • Set Value (to 0)

• Extra (Custom Variables):

  • Interpolate
  • Forward Fill
  • Backward Fill
  • Set Value (to 0)

• Macro (Economic Indicators):

  • Interpolate
  • Forward Fill
  • Backward Fill
  • Set Value (to 0)

• Media:

  • Set Value (to 0)

• Customer Experience:

  • Set Value (to 0)

• Competitor Media:

  • Set Value (to 0)

Notes

• These rules are designed to balance accuracy, computational efficiency, and business logic. For example, media-related variables default to 0 for missing data, assuming no spend or activity.
• If your data requires custom handling beyond these defaults, consider preprocessing it before upload or consulting with a data specialist.
• Always inspect your data in the Activities dashboard after upload to verify that interpolations and extrapolations align with expectations, especially before building models.