OSIsoft AF Event Frames
.NET AGENT
Overview
The OSIsoft AF Event Frames connector enables Seeq to sync conditions based on Event Frame queries from AF databases.
It is a good fit for:
Well-established AF Event Frames databases, regardless of the means of Event Frame creation
AF Event Frames databases that have a large volume of auto-generated content, especially those populated by BES/MES Interfaces
Other AF Event Frames databases that follow ISA88 structure
It is not the recommended solution for new AF Event Frames databases where the primary means of content generation is by AF Analyses that generate Event Frames.
Why not use the connector for Event Frames generated by AF Analyses? It turns out that in most cases, a combination of Value Search, Composite Condition, and Asset Swapping in Seeq will be a more efficient and robust path to generating the exact same conditions as one would create from using the Event Frames generators in the AF Management console. As such, it doesn't make sense to do double work to create the AF content, then map it into Seeq - unless a great deal of investment has already been made in building up the AF-only content.
The OSIsoft AF Event Frames connector provides two mechanisms to improve scaling - partitioning and descendant Conditions. These enable the creation of dozens or hundreds of Conditions for a single query configuration by taking advantage of the attributes and structural relationships of Event Frames in the AF Database.
Prerequisites
The AF Event Frames Connector is licensed as a Seeq Connector Plus and will not start up unless the Seeq Server installation is licensed to support it. Please contact your Seeq Sales Executive or Account Manager for more information regarding licensing.
Configuration
This is an example configuration template that is displayed in the Additional Configuration box that appears when you click Configure for an existing datasource (or if a new datasource is being created, in the Create new datasource connection modal that appears after clicking Add Datasource) on the Datasources administration page.
{
"EventFramesQueries": [
{
"ConfigName": "Sample Query Configuration",
"ConditionNameTemplate": "Condition_${EventFrameName}",
"ConditionDescriptionTemplate": "This is Condition_${EventFrameName}",
"ConditionMaxDuration": "1day",
"SearchFullHierarchy": false,
"SearchMode": "Overlapped",
"SearchRootId": null,
"EventFrameNameFilter": "*EventFrameNameEnd",
"EventFrameCategoryFilter": "CategoryX",
"ElementNameFilter": "ElementNameStart*",
"PrimaryReferencedElementCategoryFilter": "ElementL",
"EventFrameTemplateFilter": "Procedure",
"AttributeFilters": null,
"ExtendedPropertyFilters": null,
"Partitions": [
{
"PartitionName": "EventFrameName",
"PartitionScope": "Name",
"PartitionExpression": "",
"PartitionSearchFilter": ""
}
],
"CreateConditionsForDescendants": true,
"PropagatedProperties": [
"BatchID"
],
"HierarchicalConditions": false,
"TrainingIntervalStart": "2018-11-05T20:21:14.9337781Z",
"TrainingIntervalEnd": "2018-11-12T20:21:14.9337781Z",
"ResetTrainingWindow": true,
"ExtendTrainingWindowEachSync": true
}
]
}
OSIsoft AF Event Frames Additional Configuration
Property Name | Default Value | Data Type | Description |
---|---|---|---|
|
| String | The ID of the AF Server. In PI System Explorer, this is found by choosing File > Connections…, then choosing Properties from the right-click context menu for the AF server of interest. |
|
| String | Usually you will use a service account to run the agent, so this field will be left |
|
| String/Secrets File | As noted for |
| “1h” | String | Time between the end of one incremental index and the start of the next, assuming no full index is in progress. Should be of the form |
| An empty List | List | This list is populated by the agent automatically upon establishing a connection to the AF server. Each entry in the list will have fields for Only enabled databases will be indexed. |
EventFramesQuery Configuration
Property Name | Default Value | Data Type | Description |
---|---|---|---|
|
| String | The name of the configuration. This is used only for logging purposes, and may otherwise be used to provide a brief description of the purpose of this query. |
|
| String | This field must be unique for each configuration included in the EventFramesQueries list. For simple configurations (no partitioning or descendant Conditions), this will simply be the name of the condition that gets created in Seeq. When partitioning is applied through the use of the Partitions list, the name template will be updated based on the Event Frames found by the query and the rules established by the partitions. For each partition, the PartitionName must appear in the ConditionNameTemplate as |
|
| String | Analogous to the ConditionNameTemplate without the uniqueness requirement or the requirement to reference all PartitionNames. |
|
| Seeq-compatible duration string | Sets the Maximum Duration parameter for all conditions associated to this query. Updated for all conditions whenever the training window is reset (more on this below). |
|
| String | A search string like
If these considerations are not followed an error will be displayed in the Agent logs while indexing the connection. |
|
| String | If null, the value is set to “StartInclusive” so existing queries are upgraded without alteration to existing behavior. If set to “Overlapped” or anything other than null or “StartInclusive”, will be set to “Overlapped”. A search mode of “Overlapped” is likely to be suitable in most cases. Performance of data queries may suffer when “Overlapped” is used and many Event Frames match the search that are left in an In-Progress state indefinitely, since Event Frames longer than ConditionMaxDuration will be filtered out of the results but must be processed by the connector anyway. See the OSIsoft documentation for more information on search modes. |
|
| Object (see description) | Same as AttributeFilters (see Legacy filters below), aside from applying to the Event Frame's ExtendedProperties. It appears to be the case that the ExtendedProperties, which implements the C# IDictionary interface, can occasionally have duplicate keys, which wreaks havoc on the definition of capsule properties and the use of this filter. Use with caution. Note: Since it is not possible to query the Event Frames database using Extended Property filters, filtering is done after the query, in the connector - this has a performance impact in most cases. |
| List of PartitionDefinitions | A list of partition definitions that will be applied to define multiple Conditions from a single query using regular expressions with capture groups. See below. | |
| Boolean | If true, during a metadata sync, a search will be performed over the applicable training window, and any Event Frames that (a) are returned by the search and (b) satisfy the Seeq-only filters for the query will be checked for child Event Frames. Any child Event Frames discovered will be used to define child Conditions of the root Condition of the query based on the names of the child Event Frames, as illustrated in the description for ConditionNameTemplate. This is particularly useful for ISA88 hierarchies, since the nomenclature of child Event Frames will be fairly consistent from batch to batch. | |
| List of strings | When CreateConditionsForDescendants is true, capsules on child Conditions may not always have the properties needed to make good use of the Conditions in Seeq, so PropagatedProperties provides a means to transfer properties found on parent Event Frames to the capsules of the child Conditions. A common use for this is to transfer the BatchID from the UnitProcedure level to Operations, Phases, and Steps for ISA88-based Event Frames. A JSON list of strings should be specified for this parameter. Propagation is dominated by the lowest-occurring instance of the property, so if "Max Temp" is in the list of propagated properties, and a child Event Frame specifies a "Max Temp" prop that also appears on an ancestor of the Event Frame, the child Event Frame's version will apply for the associated capsule. Note: | |
| Boolean | If true, the child Conditions created when CreateConditionsForDescendants is true will be set as children of their parent Conditions in a Seeq Asset Tree. This parameter also determines whether child Conditions will have fully-qualified names or short names, e.g., | |
| ISO8601 timestamp string | Beginning of the time range used to query the Event Frames database for the purpose of defining Conditions during indexing. Only Event Frames found in the training window are used to determine capsule properties, descendant Conditions (representing child Event Frames), and partitions. | |
| ISO8601 timestamp string | End of the time range used to query the Event Frames database for the purpose of defining Conditions during indexing. | |
| Boolean | This must set to true whenever modifying the configuration or changing the time range over which searches should be performed to find Event Frames for the purpose of defining capsule properties, child Conditions, or partitions of the results of the query. When the parameter is changed to true and the config file is saved, the window specified by TrainingIntervalStart and TrainingIntervalEnd will be queried; existing Conditions will be archived; and Conditions will be created, updated, or removed from archive based on the result of the query over the training window. The training done during the index operation is batched, so the training window can be arbitrarily large. That said, it will obviously take longer to train the query for intervals that contain more Event Frames, so the initial window should be chosen to cover all variations of interest but with no greater duration than necessary. After a successful sync, the ResetTrainingWindow parameter will be set back to false. | |
| Boolean | If true, whenever an index is done, a new search will be performed from the end of the previous training window to the current time. This is useful if partitioning or child Conditions are enabled and new Conditions might result from searches of recent Event Frames after the initial training. |
PartitionDefinition Configuration
Property Name | Default Value | Data Type | Description |
---|---|---|---|
| String | The name of the partition, which must be included in the | |
| String | One of the following:
| |
| String | A regular expression with exactly one capture. It should not be bracketed by forward slashes. For different Event Frames matched by this query, it is expected that the result of the capture will differ. Each distinct result of the capture will define a new Condition where the name of the Condition is the ConditionNameTemplate with the string captured by the PartitionExpression substituted for its associated placeholder defined by the PartitionName. It follows that having more than one partition will result in the creation of a Condition for each unique combination of capture results. Note that if full-string matching is desired, one should include the ^ and $ characters in the regular expression to indicate the beginning and end of the string respectively; otherwise, a zero-length string can be captured! Thus, use "^(.+)$" as your JSON property value for the PartitionExpression. This ensures that the capture is the entire string; it also ensures that there is at least one character in any capture. | |
| String | An optional parameter that can be used in conjunction with the partition capture to improve search performance against the Event Frames database. After substitution of the capture, the PartitionSearchFilter should be valid in AF as an expression used in searching for Event Frames. |
Legacy Properties (for use with AF SDK versions less than 2.10.5)
If you’re starting out with the Event Frames Connector and have AF SDK 2.10.5 installed on the agent host (check the Help > About PI System Explorer… window in PI System Explorer), it is recommended to use the EventFrameSearch
field described above to define new queries, since it has better filtering capabilities and is easier to configure. Existing query configurations may be converted to use the EventFrameSearch
field by searching the agent logs for messages like Calling FindEventFrames with tokens: ...
. If using partitioning, you should look for such messages that also contain the string "Metadata sync" in the thread, since data requests may be using a more specific search token string than the one used to define the partitions during indexing.
Property Name | Default Value | Data Type | Description |
---|---|---|---|
| Boolean | This affects whether only "top-level" Event Frames are searched when querying AF. Users of PI System Explorer will be familiar with this as the "All Descendants" checkbox in Event Frames search dialogs. Whether to set this to true or false will depend on the other search parameters, especially the filters, and the structure of the database in question. | |
| String | When SearchFullHierarchy is set to true, this limits the search results to only those Event Frames that are descendants of the Event Frame whose UniqueID is specified here. Rarely used. | |
| String | Filter expression for Event Frame names. Follows AF wildcard rules and can be tested with an Event Frames search in PI System Explorer. | |
| String | Filter expression for Event Frame categories. Follows AF wildcard rules and can be tested with an Event Frames search in PI System Explorer. An Event Frame can have multiple Categories associated to it, so any matching Category will allow the Event Frame to pass the filter. | |
| String | Filter expression for Event Frame categories. Follows AF wildcard rules and can be tested with an Event Frames search in PI System Explorer. An Event Frame can have multiple Elements associated to it, so any matching Element will allow the Event Frame to pass the filter. | |
| String | Filter expression for Categories associated to the Event Frame's Primary Referenced Element, if one exists. Follows AF wildcard rules. An Element can have multiple Categories associated to it, so any matching Category will allow the Event Frame to pass the filter. | |
| String | Filter expression for the Template associated to the Event Frame, if one exists. This filter must follow the syntax rules for the query string of the AFElementTemplate.FindElementTemplates method of the AF SDK. As of R58, only one template match is allowed due to limitations of the earliest supported version AF SDK, 2.5.2. In future versions, multiple matches will be supported for AF SDK 2.7.5+. Until multiple matches are supported, it is not recommended to use this filter in combination with Template-based partitioning. | |
| Object (see description) | Seeq-only filters for string Attributes of the Event Frame. This should be a JSON dictionary of key-value pairs, where each key is either an exact name of an Attribute or a forward-slash bracketed regex that will match one or more Attributes of the Event Frame, and each value is (1) null, (2) an exact match for a string Attribute's value, or (3) a forward-slash bracketed regex to match an Attribute's value. If the key is a regex, a null value indicates that only the presence of a matching Attribute is necessary to pass the filter; otherwise, the value of each Attribute whose name matches the regex must satisfy the value filter, whether it is a regex or exact match. If the key is not a regex, only one Attribute with the exact-match name will exist on the Event Frame, and its value must pass the value regex or be an exact match to the specified value. |
Known Issues
Capsule Properties
An Event Frame’s properties are converted into capsule properties by the connector, e.g., Name, Template, Primary Referenced Element Name. Attributes and Extended Properties are also mapped to capsule properties, but this can lead to ambiguities; for example, an attribute named Name or Template would conflict with the Event Frame Name or Template property created by the connector.
Event Frame properties named Start, End, or Duration are mapped to capsule properties with the names Attribute: Start
, Attribute: End
, and Attribute: Duration
to avoid conflicts with the capsule properties Start
, End
, and Duration
in Seeq.
It is possible to use Property Transforms to make changes to properties. These can be used to create alternative names for properties and to remove undesired properties.
Property Ambiguity
Ambiguity is resolved when detected by prefixing an attribute- or extended property-based capsule property name with Attribute:
or Extended Property:
, respectively. Thus, an Attribute of an Event Frame named Name
will show up in Seeq as a capsule property named Attribute: Name
. This only occurs if a conflict is detected between hardcoded names and Attributes or between Attributes and Extended Properties (the latter two sets are disallowed from having duplicate names within AF).
Uncertainty
Within the OSIsoft ecosystem, a PIBatch or Event Frame is uncertain (or in OSIsoft terminology, in progress) if its end time is set to the maximum value. For Event Frames, this value is AFTime.MaxValue. OSIsoft-provided client software typically coerces this end time to the AF server time when an in-progress Event Frame is requested. Within Seeq, we take a different approach, one that is global for a Condition rather than specific to a capsule: we define uncertainty based on a capsule's start time relative to a special timestamp called the cursor. Since the definition of uncertainty is defined solely based on capsule start times and cannot be set differently for different capsules, there is currently no way to set certainty to correlate completely with the notion of "in-progress" on AF. Since we would rather be incorrect in indicating a capsule is uncertain than incorrect in indicating a capsule is certain, we choose to define the position of the Seeq cursor for a Condition indexed by the AF Event Frames Connector as the current time on the Seeq Server minus the Maximum Duration set in the config file for the Condition. Note that this Maximum Duration may diverge if a user sets the Maximum Duration differently through Seeq Workbench in the Item Properties tab. The result of this choice is that, assuming the Maximum Duration of the Condition specified in the config file is unchanged, no capsule can exist with a start time before the cursor and an end time at now or in the future. This means that all capsules within Maximum Duration of the Seeq Server time are indicated as uncertain, even if the associated Event Frame has an end time before the current time and is therefore no longer "in-progress" on the AF side. This actually makes it easy to distinguish false positives for uncertainty, since, aside from clock skew between Seeq and AF, any Event Frame with end time before now is no longer in-progress. As time passes, any capsules that don't exceed Maximum Duration will eventually be further back in time than the (updated) cursor and will render as certain in the UI.
Please report any other issues you find to our support portal.
Troubleshooting
If you are running into issues with connecting to or access data from OSIsoft AF Event Frames, view our guide for troubleshooting datasource issues.
Performance considerations
Connecting to OSIsoft AF Event Frames does not have any special performance considerations. View our guide on optimizing datasource performance for general guidance.