[Recording] TrendMiner Success Series: Admin - TrendMiner Indexing: From Understanding to Optimization

At which delay percentage will a tag get high priority?

• A tag has an index policy which determines how often it updates its recent data (forward indexing).
• An index task can be a historical index task (backward indexing) or a recent index data update (forward indexing).
• An index task has a priority.
• Priority of an index task:
  • Historical index task (backward indexing):
    • Default: normal prio
    • First 3 historical index tasks (3 most recent months): high prio
    • If in queue for 1 hour without being processed: prio is updated from normal to high prio
  • Recent index update (forward indexing):
    • Always normal prio
• The order in which recent index updates (forward indexing) are scheduled is based on their delay. 
• Index delay is calculated based on the tags index policy.
  • Example 1: default policy 1 hour with last update 80 minutes ago: 1 + (80m – 60m) / 60m => delay = 1,33
  • Example 2: monitored tag, 2 min policy with last update 5 min ago: 1 + (5m – 2m) / 2m => delay = 2,5
• Once recent index update is scheduled it stays in the queue until it is processed. 

Hopefully this clarifies a bit.

In the presentation it is mentioned that there are 2 types of priority (normal, high). When does become high priority, from which delay?

The delay does not influence the priority. 

Every 5 seconds the index scheduler calculates which tags have the highest delay. The ones with the highest delays are added to the queue for an update, but always with normal priority. Once a task is in the queue it stays there until it’s processed.

So technically speaking the priority never changes.

Note that this applies to forward indexing (adding new data).

Index tasks can only have a high prio in the following cases:

• The 3 most recent backward index tasks (a tag which gets used/indexed for the first time)
• A backward index tasks which is in the queue for 1 hour changes from normal to high prio
• An interactive index update: this is when a user is actively using the tag. When a user selects the tag it forces an index update for that tag. This index update has a high prio.