Troubleshooting

Overview

When you are running into issues with the ADX connector there are a number of places that you can go to identify the cause.

Identifying Issues Through the Datasource Log

When there are issues with a specific query, you can navigate to the connection log. From the Datasources tab within the Administration page, click on view log in the datasource card:

You can filter by Error in the filters bar at the top:

Errors

Reading Error messages

com.microsoft.azure.kusto.data.exceptions.DataClientException: Error in post request

Typically this will be followed by a Caused By trace that will identify the actual error. For example: Caused by: org.apache.http.conn.ConnectTimeoutException.

Common Issues

org.apache.http.conn.ConnectTimeoutException

There is either a problem with your ADX cluster, or your network link between the Remote Agent where the connector is running and the ADX cluster.

com.microsoft.azure.kusto.data.exceptions.DataWebException

This error will have an error code associated with it. Depending on the code there are different troubleshooting steps.

Too many requests

CODE

"error": {
        "code": "Too many requests",
        "message": "Request is denied due to throttling.",
        "@type": "Kusto.DataNode.Exceptions.ControlCommandThrottledException"

The ADX cluster is throttling the requests as there are too many concurrent requests. If this is a heavily used cluster consider increasing the cluster size or increasing the cluster concurrency. See more information here: https://learn.microsoft.com/en-us/azure/data-explorer/high-concurrency . If there are a high number of data requests, consider enabling persistent caching on the datasource within Seeq.

General_BadRequest

CODE

"error": {
        "code": "General_BadRequest",
        "message": "Request is invalid and cannot be executed.",
        "@type": "Kusto.Data.Exceptions.KustoBadRequestException",
        "@message": "Request is invalid and cannot be processed: Semantic error: SEM0100: 'project' operator: Failed to resolve scalar expression named 'wrongId'",

The query is invalid. In this case the @message will tell you what is wrong with the query.

java.io.IOException: Error, no assets or signals were indexed. Please check configuration

In this case it will be typical to see an error right before this message. For example: com.seeq.link.connectors.adx.AdxConnection - Error, cannot index table id: ETV Data 1 due to incorrect configuration. SignalPropertiesMap is required when MetadataQuery["TableName"] is supplied. If there is no error message then there is a chance that the query in either the MetadataQuery or ComputedGroupByResult is incorrect, or that the NameQuery or DataColumnQuery is returning nothing.

java.lang.IllegalStateException: <path> does not match table definition <table definition name> - groupBy path 'path' does not match the assetpath of the signal: <signal id>

This is caused by a mismatch in the GroupBy vs the columns in the MetadataQuery or the columns returned in the ComputedGroupByResult parameter. Ensure that these parameters are set correct and are returning the information you expect.

Query execution has exceeded the allowed limits: The results of this query exceed the set limit of 500000 records

This generally happens during indexing when not using MetadataQuery. By default, ADX has a limit of 500,000 rows that it can return. when performing the distinct query for all of the columns listed in the GroupBy, there is a chance that a use case could exceed 500,000 rows. If that is the case the GroupByRequestProperties parameter will have to be set with a value of {"notruncation" : "true"}. This will allow ADX to return more than the default 500,000 rows per query.

Data queries are automatically limited and should not generate this error.

Querying Kusto

Utilizing the .show queries operator within the ADX query portal is useful to determine what queries are failing. Using a query like below:

CODE

.show queries 
| where User == "AAD app <AAD AppId from the ADX configuration>"
| where State == "Failed"
| sort by StartedOn desc 
| take 100

will show you the last 100 failures made by the Seeq application. This is useful for queries that are not executing correctly. Note, this will not include connectivity errors such as ConnectTimeoutException as ADX will not know that the query was even attempted.