Analyze performance logs

Introduction

If your Kepion application feels slower than expected, performance logs can help you understand why. Kepion’s performance recording feature captures detailed metrics about key events as users interact with the system. These metrics provide visibility into operations such as form refreshes, rule execution, writeback processes, and external pipeline executions—common areas that can impact performance.

By reviewing performance logs, you can more easily troubleshoot issues, identify bottlenecks, and optimize your application’s responsiveness.

Before you start

• Ensure that performance logging is enabled for your application. See details here. 

• Configure the appropriate log retention period. See details here.

Explore the performance log schema

Kepion stores performance data in several log tables that can be queried for diagnostics and analysis.

[log].[PerformanceLogs]

Provides a high-level summary of each operation logged.

The table below explains each column:

[log].[PerformanceLogDetails]

Stores detailed information for each operation.

The table below explains each column:

[log].[vwPerformanceLogs]

A view that joins summary logs with detailed log entries.

SELECT
   PL.[LogId]
  ,PL.[Operation]
  ,PL.[UserId]
  ,PL.[CreatedOn]
  ,PL.[DurationSeconds]
  ,PLD.[Key]
  ,PLD.[Value]
FROM [log].[PerformanceLogs] PL
INNER JOIN [log].[PerformanceLogDetails] PLD
  ON PL.[LogId] = PLD.[LogId];

Understand the performance metrics

Performance logs capture different types of operations. Each operation includes a set of keys that describe additional context and processing details.

Form performance

Operations

• refresh_form
  • Form data refresh operations
  • Captures MDX query execution time
  • Tracks cell count and processing breakdown
  • Records form-specific context

Keys

• form_id - Form identifier
• mdx_query - The MDX query executed
• cell_count - Number of cells processed
• duration_breakdown - JSON breakdown of processing time

{     
  "mdx_duration": 1.25,     
  "annotations_duration": 0.15,     
  "dropdowns_duration": 0.08   
}  

Rule performance

Operations

• run_rule - General rule execution
• run_rule_deploy_mdx_rules - MDX rule deployment
• run_rule_deploy_security - Security deployment
• run_rule_deploy_solution - Solution deployment
• run_rule_process_database - Database processing
• run_rule_process_dimension - Dimension processing
• run_rule_process_partition - Partition processing

Keys

• activity_id - Activity identifier for tracking
• rule_id - Rule identifier
• process_type - Type of processing performed
   

File processing performance

Operations

• delete_fs_file - File deletion from file store
• download_fs_file - File download operations
• upload_fs_file - File upload operations

Keys

• file - File path or name
• size_kb - File size in kilobytes

Writeback performance

Operations

• writeback_data - Data writeback operations
• run_breakback - Breakback rule execution

Writeback-specific keys

• model_id - Model identifier
• records_count - Number of records processed

Azure Data Factory pipeline performance

Operations

• run_adf_pipeline - Azure Data Factory pipeline execution.

ADF pipeline-specific keys

• integration_uid - Identifier of the Kepion integration used to execute the pipeline.
• pipeline_start_date - UTC timestamp indicating when the pipeline execution started in Azure Data Factory.
• run_id - Azure Data Factory pipeline run identifier. This can be used to correlate the Kepion performance log with the corresponding execution in ADF.
• status - Final execution status of the pipeline, such as Succeeded, Failed, or Cancelled.
• error - Error information associated with the execution. Present when the pipeline fails, is cancelled, or Kepion cannot start/resolve the requested pipeline.

Configure log settings

Log settings are configured in the [log].[Settings] table.

Example queries

Basic performance overview

SELECT
    Operation,
    COUNT(*) as ExecutionCount,
    AVG(DurationSeconds) as AvgDuration,
    MAX(DurationSeconds) as MaxDuration,
    MIN(DurationSeconds) as MinDuration
FROM [log].[PerformanceLogs]
WHERE CreatedOn >= DATEADD(day, -7, GETUTCDATE())
GROUP BY Operation
ORDER BY AvgDuration DESC;

Slow operations (Top 10)

SELECT TOP 10
    pl.Operation,
    pl.DurationSeconds,
    pl.CreatedOn,
    pl.UserId
FROM [log].[PerformanceLogs] pl
WHERE pl.CreatedOn >= DATEADD(day, -1, GETUTCDATE())
ORDER BY pl.DurationSeconds DESC;

User activity summary

SELECT
    pl.UserId,
    COUNT(*) as OperationCount,
    AVG(pl.DurationSeconds) as AvgDuration,
    SUM(pl.DurationSeconds) as TotalDuration
FROM [log].[PerformanceLogs] pl
WHERE pl.CreatedOn >= DATEADD(day, -7, GETUTCDATE())
GROUP BY pl.UserId
ORDER BY TotalDuration DESC;

Hourly performance trends

SELECT
    DATEPART(HOUR, pl.CreatedOn) as HourOfDay,
    pl.Operation,
    COUNT(*) as ExecutionCount,
    AVG(pl.DurationSeconds) as AvgDuration
FROM [log].[PerformanceLogs] pl
WHERE pl.CreatedOn >= DATEADD(day, -7, GETUTCDATE())
GROUP BY DATEPART(HOUR, pl.CreatedOn), pl.Operation
ORDER BY HourOfDay, AvgDuration DESC;

Form performance analysis

SELECT
    pld.Value as FormId,
    pl.Operation,
    pl.DurationSeconds,
    pl.CreatedOn,
    pld2.Value as CellCount
FROM [log].[PerformanceLogs] pl
INNER JOIN [log].[PerformanceLogDetails] pld
    ON pl.LogId = pld.LogId AND pld.[Key] = 'form_id'
LEFT JOIN [log].[PerformanceLogDetails] pld2
    ON pl.LogId = pld2.LogId AND pld2.[Key] = 'cell_count'
WHERE pl.Operation = 'refresh_form'
    AND pl.CreatedOn >= DATEADD(day, -1, GETUTCDATE())
ORDER BY pl.DurationSeconds DESC;

Rule execution performance

SELECT
    pl.Operation,
    pl.DurationSeconds,
    pl.CreatedOn,
    pld.Value as RuleId,
    pld2.Value as Context
FROM [log].[PerformanceLogs] pl
LEFT JOIN [log].[PerformanceLogDetails] pld
    ON pl.LogId = pld.LogId AND pld.[Key] = 'rule_id'
LEFT JOIN [log].[PerformanceLogDetails] pld2
    ON pl.LogId = pld2.LogId AND pld2.[Key] = 'context'
WHERE pl.Operation LIKE 'run_rule%'
    AND pl.CreatedOn >= DATEADD(day, -7, GETUTCDATE())
ORDER BY pl.DurationSeconds DESC;

Error analysis

SELECT
    pl.Operation,
    pl.DurationSeconds,
    pl.CreatedOn,
    pld.Value as ErrorMessage
FROM [log].[PerformanceLogs] pl
INNER JOIN [log].[PerformanceLogDetails] pld
    ON pl.LogId = pld.LogId AND pld.[Key] = 'error'
WHERE pl.CreatedOn >= DATEADD(day, -1, GETUTCDATE())
ORDER BY pl.CreatedOn DESC;

ADF Pipeline execution analysis

SELECT
    pl.LogId,
    pl.DurationSeconds,
    pl.CreatedOn,
    MAX(CASE WHEN pld.[Key] = 'integration_uid'
             THEN pld.[Value] END) AS IntegrationUid,
    MAX(CASE WHEN pld.[Key] = 'run_id'
             THEN pld.[Value] END) AS RunId,
    MAX(CASE WHEN pld.[Key] = 'pipeline_start_date'
             THEN pld.[Value] END) AS PipelineStartDate,
    MAX(CASE WHEN pld.[Key] = 'status'
             THEN pld.[Value] END) AS Status,
    MAX(CASE WHEN pld.[Key] = 'error'
             THEN pld.[Value] END) AS Error
FROM [log].[PerformanceLogs] pl
LEFT JOIN [log].[PerformanceLogDetails] pld
    ON pld.LogId = pl.LogId
WHERE pl.Operation = 'run_adf_pipeline'
    AND pl.CreatedOn >= DATEADD(day, -7, GETUTCDATE())
GROUP BY
    pl.LogId,
    pl.DurationSeconds,
    pl.CreatedOn
ORDER BY pl.CreatedOn DESC;