Advanced Configuration

Data Lab Functions has various kernel and log configuration properties that can be set per Add-on project.

Configuration Properties

kernel_name

Unlike a typical Jupyter notebook where the notebook kernel is determined from the kernel name stored in the notebook metadata, Data Lab Functions uses the kernel_name configuration property in determining the actual kernel language and version to launch when the Data Lab Functions project starts. Valid values are python38 for Python 3.8 kernel and python311 for Python 3.11. If unspecified the default kernel is the oldest Python version currently installed in Data Lab. See Python Versions and Packages for currently supported Python versions.

kernel_global_pool_min

When a Data Lab Functions project starts, kernels are immediately launched and bootstrapped with Data Lab Functions initialization code (various Python package imports) so that these kernels can be “warmed” in anticipation of actual Data Lab Functions requests. Once a request is received, a kernel from this “global kernel pool” will be assigned to the Seeq user making the request. A new kernel will be started and added to the global poll in its place. The kernel_global_pool_min property determines the number of kernels to maintain in the global kernel pool through the life of the Data Lab Functions project instance. If unspecified, the default minimum number of global kernels is 5.

kernel_user_pool_max

Data Lab Functions recycles kernels from previous requests by assigning the kernel to a user-specific kernel pool. When a new request is received, a kernel from this user will pool will be used to service the request. If none exist or are busy servicing other requests, a kernel will be adopted from the global kernel pool and assigned to the current user. The kernel_user_pool_max property is the maximum number of simultaneously executing kernels for a given user. If unspecified, the default maximum number of user kernels is 10.

kernel_cull_interval

When there is inactivity for requests to Data Lab Functions, kernels are culled to reduce overhead and resources. The kernel_cull_interval property is the frequency in seconds to run the process to check for inactive kernels to destroy. If unspecified, the default frequency to check for inactive kernels is 60 seconds.

kernel_cull_idle_timeout

When there is inactivity for requests to Data Lab Functions, kernels are culled to reduce overhead and resources. The kernel_cull_idle_timeout property is the number of seconds a kernel must be inactive to be a candidate for culling. If unspecified, the default frequency to check for inactive kernels is 120 seconds.

log_level

The log level to display Data Lab Functions Add-on log messages. Accepted values are TRACE, DEBUG, INFO (default), WARNING, ERROR. The default log_level value is INFO.

log_format

The formatter for the Data Lab Functions Add-on log messages. The default log_format is %(levelname)-7s %(asctime)s.%(msecs)03dZ %(name)s - User: %(user_id)s %(notebook)s %(method)s %(endpoint)s - Request: %(request_id)s :: %(message)s.

log_when

The frequency which to rotate the Data Lab Functions Add-on log files. Accepted values are MINUTE, HOUR, DAY, or MIDNIGHT. The default log_when value is HOUR.

log_when_interval

The integer interval for the log rotation frequency. The default log_when_interval value is 1.

log_backup_count

The number of Data Lab Functions Add-on log backup files to keep before rotating the log file. The default log_backup_count value is 24.

Configuration Endpoints

Configuration for a Data Lab Functions Add-on project can be retrieved or set through REST endpoints. for users with manage permission to the project.

GET functions/config

The GET endpoint retrieves all configuration properties for the current instance of the Data Lab Functions Add-on project.

POST functions/config

The POST endpoint updates the configuration of the current instance of the Data Lab Functions Add-on project. Restarting the project will revert the configuration settings to the values applied to the Add-on project as set in Add-on Manager. With the exception of inactivity_timeout and kernel_name, all configuration settings can be updated for a running instance:

An example of the body schema for the POST endpoint:

{
  "configuration": [
    {
      "name": "kernel_cull_interval",
      "value": 60
    },
    {
      "name": "kernel_cull_idle_timeout",
      "value": 120
    },
    {
      "name": "kernel_global_pool_min",
      "value": 5
    },
    {
      "name": "kernel_name",
      "value": "python3"
    },
    {
      "name": "kernel_user_pool_max",
      "value": 10
    },
	{
      "name": "log_level",
      "value": "INFO"
    },
	{
      "name": "log_format",
      "value": "%(levelname)-7s %(asctime)s.%(msecs)03dZ %(name)s - User: %(user_id)s %(notebook)s %(method)s %(endpoint)s - Request: %(request_id)s :: %(message)s"
    },
	{
      "name": "log_backup_count",
      "value": 5
    },
	{
      "name": "log_when",
      "value": "midnight"
    },
	{
      "name": "log_when_interval",
      "value": 1
    }
  ]
}