Skip to main content
Skip table of contents

Add-on Calculation Engine

Overview

The Add-on Calculation Engine allows customers to extend the calculation capabilities of Seeq with the compute capabilities of Python.

What can it do?

Add-on Calculations allows users to run Python scripts on Seeq Signals. The results of the calculation are directly accessible as Signals within Seeq Formula.

These calculations can either be performed by key or by sliding-window. Both types of calculation produce a Signal as output.

Key calculations give a Python script access to the sample (or samples, if multiple signals) at a particular key. The script then performs calculations to produce an output sample at that key.

Sliding-window calculations give a Python script access to a sliding-window of keys and all samples present at these keys. The script then performs calculations to produce an output keyed at the start of the sliding window.

How it works

The Python Connector watches for script packages in a folder named user on Seeq Remote Agents.

For Seeq SaaS, Remote Agents are the preferred location for add-on calculation scripts. For legacy on-prem, it is also possible to place them on the Seeq Server.

This folder is located at ~/.seeq/data/add-ons/calculations/python/user on Linux or MacOS, and at C:\ProgramData\Seeq\data\add-ons\calculations\python\user on Windows.

User File location on a Windows Seeq Server

User File location on a Windows Seeq Server

The connector looks for changes in the user folder, appropriately responding to package creation, updates and removal.

The example folder contains an example package called Seeq_Examples that can be used as a starting point and inspiration for your own custom packages.

All changes in the example folder will get overwritten when Seeq restarts.

The extcalc folder is a Python module where the calculation engine and helping classes are provided.

readme.html provides more detailed descriptions of examples as well as the specific coding requirements of Add-on Calculations.

Script Packages

Each folder in the user folder will create a User Defined Function package (see User-Defined Formula Functions ). These can be used to organize your scripts, such as by team or by topic.

Three packages in the user folder - Forecasting, RootCause, and SiteA

Three packages in the user folder - Forecasting, RootCause, and SiteA

When creating a new package, create the package folder first, then add scripts inside of it. Copying an entire package folder containing scripts into the user folder can occasionally cause scripts to not be integrated properly.

Package names are used as a prefix in the created User Defined Functions. For example, all scripts in the Forecasting package will produce functions with names such as Forecasting_function1, Forecasting_function2, etc. This makes searching for them in Formula Documentation easy: all you need is the package name to find all scripts in that package.

Script Definitions

It is recommended to start from an example script from examples/Seeq_Examples. Create a new folder inside of the user folder to serve as your package. Copy the example script into your new package folder and begin modifying the compute, compute_output_mode and function_details methods. There is no need to restart Seeq when the script is changed as the entire user folder and its children are watched continuously for changes.

Python Script Details

Python Script Details

The file name of your script will be used as the name of the function created in Seeq. The script must end in the .py extension. It must be placed in a package folder, which is itself in the user directory. Multiple scripts can live in the same package folder, but they must have different Python files. For example

The Forecasting package containing two scripts - Function1.py and Function2.py

The Forecasting package containing two scripts - Function1.py and Function2.py

Each Python file must have a class with the same name as the file. This class should extend one of KeywiseExternalCalculationScript or WindowExternalCalculationScript, depending on if your script operates on data at a single key, or over a window of data. Both of these classes support the following methods:

Method Name

Purpose

Called when?

parameters

return values

Required

initialize

Initialize any values needed by the script.

once, at the very beginning of execution

none

none

No

validate

Validate that input types are correct.

once, right after initialize

validation_object

none

Yes

compute_output_mode

Tell Seeq if this Add-on calculation will return a String Scalar or numeric Scalar.

once, right after validate

none

‘NUMERIC’ or ‘STRING’

Yes

function_definition

Holds the User Defined Function information for this Add-on calculation.

once for each execution

none

{function_details} see below.

No

cleanup

Clean up any state created by the script.

once, at the end of execution

none

none

No

get_test_signals_data_types

Define the types of data used to test with get_test_data.

once, when running in test mode

none

[test_data_types], all lists elements are ‘STRING’ or ‘NUMERIC’

No

get_test_data

Return test data for testing the script.

once, when running in test mode

none

[test_data] see below.

No

Additionally, there is a compute method that differs between the two classes, as well as a get_test_window_size method only for WindowExternalCalculationScripts.

Method Name (class)

Called when?

parameters

return values

Required

compute (Keywise)

once for each key in the input signals

key, samples_for_key

value at key

Yes

compute (Window)

once for each key in the input signals

keys, samples_for_keys

value at first key of window

Yes

get_test_window_size

once, when running in test mode

none

int

No

Most scripts will only overwrite the function_details, validate, compute_output_mode and compute methods. Of these, the two computation methods and function_details are required, and validate is optional.

Invalid or missing samples in inputs are represented as None. You can use extcalc.Invalid to return a sample with Scalar.INVALID or extcalc.NoValue if the calculation does not produce a value at that key.

Test Data

Data for testing your python script is an array of arrays. The first value in each array is the key value (keys in Seeq are epoch nanosecond longs). Other values in each array are the values of signal inputs at that particular key.

CODE
For example, for keys 1, 2, 3, 4 and two signals where the
first signal is NUMERIC and with values 2, 4, 6, 8 and
the second signal is STRING with values 'red', 'blue', 'green',
'white', the input would look like this:
[
  [1, 2, 'red'],
  [2, 4, 'blue'],
  [3, 6, 'green'],
  [4, 8, 'white'],
]

Function Details (User Defined Function information)

Function Details

This portion defines the User Defined Function that will be created in Seeq to represent this add-on calculation. It returns a dictionary with five components: Name, Documentation, Formula, Parameters, and Examples.

Name (String)

The name of the function. This name will be used to invoke the Add-on Calculation inside Seeq Formula.

Documentation (String)

A written description of your function. Can include any level of detail you desire. Optional.

Formula (String)

The formula of the created User Defined Function. Allows to pre-process the input signals using Seeq functions, or post-process the signal result of your Python calculation. The simplest form for a function with one parameter named $signal is addOnCalculation(@@scriptId@@, $signal).
This Formula must contain a call to addOnCalculation or addOnCalculationOnWindow with @@scriptId@@ as the first parameter. Input signals can be used by other operators within this Formula, or passed directly to the Python calculation.

Parameters (Array of Dictionary of Strings)

Each entry is a parameter to the created User Defined Function. Note that not all parameters must be passed to the Python script: some can be used in the Formula without passing them on. There are four fields settable in each parameter. At least one parameter is required in all Add-on Calculations.

Parameters in the list must be ordered

  1. Mandatory parameters.

  2. Optional parameters with default values.

  3. Optional parameters without default values.

Optional parameters let you create different variants of your Add-on Calculation that can be called with different numbers of parameters.

Field Name

Description

Examples

Required

Name

Name of the parameter.

signal, throttle, input1

Yes

Formula

A formula used to determine the type (signal, condition, capsule, scalar) and metadata (unit of measure) of the parameter. The actual calculation of the formula is unimportant, only the type and units of the formula are used.

If you are using default values, formula should NOT be set.

1m.toSignal() - parameter must be a signal with unit of measure meters.

days() - parameter must be a condition.

capsule(42s, 42s) - parameter must be a capsule

42 - parameter must be a unitless scalar.

Yes (if no default value)

Optional

A boolean flag describing if this parameter must be included in calls to the Add-on Calculation. If this field is not set, it is assumed that this parameter is required.

true

No

DefaultValue

A default value to be used for this parameter if none is provided. Should be a valid Seeq formula. This field should only be set if Optional is set to true. If this field is set, you should NOT set the formula for this parameter.

42m

sinusoid()

No

Either the Formula or DefaultValue fields are used to compute the type and unit of measure of a parameter. Set exactly one of them: not both, and not neither.

Examples (Array of Dictionary of Strings)

Each entry is an example of the usage of your Add-on Calculation. Examples are Optional.

An example is a dictionary that can have two fields. If present, these examples will be visible in the Formula documentation.

Field Name

Description

Examples

Required

Formula

The formula of your example. Use @@functionName@@ as a placeholder for the name of the Add-on Calculation. It will get replaced by packageName_functionName where packageName comes from the Package (see section above) and functionName comes from the name field of Function Details.

@@functionName@@($seriesA, $seriesB, 2.5, 2)

Yes

Description

A description of the example.

Multiplies $seriesA by $seriesB, scaled by 2.5 and adds 2.

No

Different examples can be used to document different variants of the Add-on Calculation (invocations with different sets of optional parameters supplied).

Testing Your Scripts

This sections is for testing the python portion of your calculation. To test the entire Add-on Calculation, try using it in Seeq Formula.

  1. Override method get_test_signals_data_types in your class. This should return an array with the data types of the test input signals. You can use your own implementation or the predefined classes which provide test data.

  2. Override method get_test_data in your class. This should return an array with data to be used for testing. You can use your own implementation or the predefined classes which provide test data. Details about test value format can be found in the class documentation of the script files or in the example scripts.

  3. In case of a window based computation, override get_test_window_size also.

  4. Start the script using python -m extcalc <ClassName> <PackageFolderName> --test in the <seeq-data-dir>/add-ons/calculations/python directory.

FAQ

Q: It doesn’t seem like Seeq is loading my function. What can I do?
A: Make a test Package and copy over one of the Seeq_Example python files into it. If this fails to load properly, there may be some issues with the Add-on Calculation connector. If this succeeds, your script likely has some syntax errors. Try testing the python script using the information in Testing Your Scripts above. Confirm that all parts of your Function Details are correct. Take a look into the jvm-link logs, searching for “Calculation” to find potentially helpful error messages.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.