Optimizing for Infrequently Updating Data
This document explains the background about resampleHold(), a new function in Seeq Formula in v60 designed to help optimize performance for infrequently updating data.
Infrequently updating data are signals that don't necessarily come from a physical sensor, where the last sample might be a week or months ago. This signals have many different names: lab data, boundary values, run states, etc.
For example, let's examine how much time a signal spends above the set point.
Notice how the uncertainty of the set point signal starts at the last sample (Feb 3), and then all the capsules after that are considered uncertain. Uncertain data can’t be cached, so these regions of uncertainty can have a negative impact on performance by requiring the computation to be repeated.
This is at odds with user expectations about set point signals. The value is meant to be in effect until the next sample is entered, probably some time in the future, but definitely not significantly in the past.
If we create an altered signal using the following formula, we get a much more certain result:
$setpoint.resampleHold(60d)
Function Details
The function has the following parameters:
$signal.resampleHold($holdDuration [, $period])The $holdDuration is the primary feature, where it will look back that amount of time to find the last sample, and create new samples each $period using the previous value. The $period is optional, with a default of 10 minutes. The original sample is unaltered, this just creates new samples.
Not just near now
This function is usually used for extending certainty closer to now, but it's also useful for interpolating large gaps in historical data. Previously, you may have used the Maximum Interpolation property (or its formula equivalent, $signal.setMaxInterpolation($value)) to configure a signal to be interpolated between any two samples less than $value apart. If your data was farther apart than the maximum interpolation, a gap would appear.
In this data, the 3 lanes show the differences:
Discrete data, displayed as bars to be obvious where the samples are.
A step signal of that data with a 60 day max interpolation. Note the big gap in the middle where samples are almost 3 months apart
With
resampleHold(60d), it interpolates 60 days of the gap.
Troubleshooting
If you've added this new function to your formula and the uncertainty remains at the last sample, it generally means the existing uncertainty is more pessimistic than necessary. This function does not change the uncertainty of the signal, it only changes how much of the signal can be interpolated with certainty.
At the datasource
Some connectors default their uncertainty to be at the last sample. This can be overriden by a Seeq Admin using the Uncertainty Override property on a signal (see the Optimizing a Datasource Connection). This is often done with connector transforms in the connection configuration.
For example, with an uncertainty override of 1 hour, the administrator is declaring that it never takes more than a hour to propagate data from the sensor to the historian. Put another way, it's saying the historian won't rewrite history on data more than an hour old, marking it safe to cache.
Seeq as datasource
Some signals are stored directly in Seeq, such as CSV imports, data pushed from Seeq Data Lab, or even formulas using signal(). These signals also place the uncertainty at the last sample. You can mimic the uncertainty override using formula:
$signal.setUncertaintyOverride($lagFromNow)