About Str:::lab Studio

Str:::lab Studio is a browser-based SQL IDE that connects directly to any Apache Flink SQL Gateway. It provides a complete development, monitoring, and administration experience — without the overhead of a full platform deployment. Works with local Docker, Kubernetes, and any self-hosted or managed Flink cluster.

Quickstart

git clone https://github.com/coded-streams/strlabstudio
cd strlabstudio
docker compose up -d
open http://localhost:3030

python3 -m http.server 8080
# Open studio/index.html — use Direct Gateway mode

helm repo add strlabstudio https://coded-streams.github.io/strlabstudio-operator/charts
helm install strlabstudio-operator strlabstudio/strlabstudio-operator \
  --namespace flinksql-system --create-namespace

kubectl apply -f - <<EOF
apiVersion: codedstreams.io/v1alpha1
kind: StrlabStudio
metadata:
  name: my-studio
  namespace: flink
spec:
  gateway:
    host: flink-sql-gateway
    port: 8083
  jobmanager:
    host: flink-jobmanager
    port: 8081
EOF

Connection Modes

Architecture

Str:::lab Studio is a static HTML application served by nginx. All SQL execution happens via the Flink SQL Gateway REST API. The browser manages all session state client-side.

State Management

All Features

Editor

Sessions

Job Graph

Performance Monitoring

Snippets Library

30+ built-in SQL templates accessible from the toolbar. Click any snippet to insert into the active editor tab.

Project Manager

Save, load, run, and organise named Str:::lab projects via Projects in the topbar. Each project stores SQL tabs, SET config, catalog context, run count, and timestamps. Export and import as JSON files.

UDF Manager

Complete UDF lifecycle management: browse registered functions and views, upload JARs directly to the cluster, register functions (Java/Scala/Python — LANGUAGE SQL does not exist in Flink and has been removed), build SQL expressions via the View Builder, generate Maven/Gradle configs, and insert professional UDF templates. Fixes in v2: SHOW FUNCTIONS now returns all rows, library search works across Views and UDFs, View Builder Insert Expression works correctly when the form is filled.

Colour Describe

User-defined live row highlighting. Build rules (field + operator + value + color + style mode) that apply to streaming result rows in real time. Operators: == != > >= < <=, contains, starts, ends, regex. Style modes: row background, left border accent, text color. Rules evaluated top-to-bottom — first match wins. Color legend rendered below the table with a Turn off button. Report integration included.

Pipeline Manager v2.1

Visual drag-and-drop Flink SQL pipeline builder. Drag operators from the palette onto the canvas, connect them, configure parameters inline, and get production-ready Flink SQL generated automatically. Access via Managers → Pipeline Manager.

Operator Palette

Edge Types

Canvas Controls

• Drag nodes from the palette onto the canvas. Nodes can be freely repositioned by dragging.
• Connect nodes by dragging from an output port (right edge) to an input port (left edge).
• Configure by double-clicking a node — the config modal opens pinned beside the node.
• Pan by dragging the canvas background. Zoom with the mouse wheel.
• Auto Layout arranges all nodes in a clean left-to-right topology.
• Animate with the Run button — particles flow along edges for RUNNING pipelines.

Feature Engineering Manager New

A six-step guided wizard for building real-time Flink SQL feature pipelines from streaming data. Supports Schema Registry integration or manual column entry. Generates production-ready Flink SQL with per-column transforms, multiple window aggregations, configurable sinks, and a fully interactive pipeline canvas. Access via Managers → Feature Engineering.

Six-Step Wizard

Pipeline Canvas

The Review step renders an SVG-based pipeline canvas showing all four stages: Source feature columns, Transformations (with operator icons), Windows/Aggregations, and the Output node. The canvas supports:

• Scroll to zoom — mouse wheel zooms toward the cursor position, range 15%–400%
• Drag to pan — click and drag moves the viewport freely in all directions
• Fit to view — the Fit button (and double-click on background) auto-centres and scales all content
• Maximize/restore — the expand button takes the canvas to full-screen (fixed overlay) and restores it cleanly
• Zoom label — shows current zoom percentage in the toolbar

History

Every completed pipeline is saved to localStorage with the source table name, output table name, sink type, column counts, window counts, the full generated SQL, and the complete output column schema (with correctly inferred types per transformation). The History button shows up to 20 entries with an Insert SQL button per entry. History can be cleared individually or entirely.

Validation

Each step enforces required fields before allowing progression. Required fields vary by step: source table name and at least one column are required at Step 1; window time column and aggregations are required at Step 4; output table name is required at Step 5. A red validation banner appears at the top of the wizard with the specific error. Navigation via step numbers respects the same validation rules.

Generated SQL Structure

-- Feature Engineering Pipeline
-- Source: transactions_raw  Sink: user_features_out
-- Generated by Str:::lab Studio — Feature Engineering Manager

SET 'execution.runtime-mode' = 'streaming';
SET 'parallelism.default' = '4';
SET 'execution.checkpointing.interval' = '10000';
SET 'table.exec.state.ttl' = '3600000';

CREATE TEMPORARY TABLE IF NOT EXISTS transactions_raw (
  user_id                        BIGINT,
  merchant_id                    BIGINT,
  amount                         DOUBLE,
  currency                       STRING,
  status                         STRING,
  event_time                     TIMESTAMP(3),
  WATERMARK FOR event_time AS event_time - INTERVAL '5 SECOND'
) WITH (
  'connector'                    = 'kafka',
  'topic'                        = 'transactions',
  'properties.bootstrap.servers' = 'kafka:9092',
  'properties.group.id'          = 'feature-pipeline-group',
  'scan.startup.mode'            = 'latest-offset',
  'format'                       = 'json'
);

-- Window 1: TUMBLE — 5 MINUTE feature aggregation
INSERT INTO user_features_out
SELECT
  user_id,
  window_start,
  window_end,
  COUNT(*) AS tx_count_5m,
  SUM(amount) AS total_amount_5m,
  AVG(amount) AS avg_amount_5m
FROM TABLE(TUMBLE(TABLE transactions_raw, DESCRIPTOR(event_time), INTERVAL '5 MINUTE'))
GROUP BY user_id, window_start, window_end;

-- Direct feature transformation stream (no aggregation)
INSERT INTO user_features_out
SELECT
  user_id,
  ROUND(amount, 2) AS amount,
  MD5(CAST(user_id AS STRING)) AS user_id_hash,
  UPPER(status) AS status,
  CASE
    WHEN amount < 100 THEN 'bucket_0'
    WHEN amount < 500 THEN 'bucket_1'
    ELSE 'bucket_2'
  END AS amount_bucket
FROM transactions_raw;

AI Model Inference Manager New

A seven-step guided wizard for configuring real-time ML model inference on Flink streaming pipelines. Supports 25 model server types across five categories. Generates production-ready Flink SQL with async UDF patterns, all credentials embedded, and server-specific configuration documented as structured comments. Access via Managers → Inference Manager.

Source Table — Schema Auto-Population

Step 1 presents a combined table selector: a dropdown listing all known tables (Feature Engineering outputs first, then session tables), quick-pill shortcuts, a manual text input, and a From FEM button. Selecting a table auto-populates the schema using three strategies in order of richness:

1. Reads outputColumns directly from the FEM history entry — the richest source, set by the Feature Engineering Manager with correctly typed columns per transformation
2. Falls back to the strlabstudio_fem_state snapshot written at FEM pipeline save time
3. Falls back to parsing the CREATE TABLE DDL from the stored SQL string using a regex matched against the exact table name

Loaded columns are displayed as interactive tags and simultaneously populate the textarea for manual editing. If no FEM history is found for the selected table, a yellow indicator prompts the user to enter the schema manually.

Model Servers — 25 Supported

Authentication Methods

Inference I/O Configuration

• Input feature selection — click-to-toggle column tags from the loaded schema. Primary input column or multi-feature ARRAY expression for the UDF call.
• Pre-processing expression — optional SQL expression applied before the model call (e.g. CAST(amount AS DOUBLE) / 1000.0)
• Passthrough columns — columns carried forward alongside the prediction result
• Output alias and SQL type — the name and type of the prediction column (DOUBLE, STRING, ARRAY<DOUBLE>, ROW, MAP, etc.)
• Post-processing expression — optional SQL expression applied to the raw model output (e.g. CASE WHEN for classifying a score)
• Async parallelism — number of concurrent in-flight model requests per subtask
• Timeout and retry — request timeout in milliseconds and retry count on failure
• Error default — value emitted when the model call fails (null propagates the error)

Generated SQL — Credentials Included

All values entered across the wizard are emitted into the generated SQL. No placeholder values are used for fields that have been filled in. The structure is:

-- Real-time ML Inference Pipeline
-- Model Server  : AWS SageMaker
-- Auth          : aws_sigv4
-- Source Table  : user_features_out
-- Output Sink   : kafka → user_features_scored
-- Prediction    : fraud_score (DOUBLE)

SET 'execution.runtime-mode' = 'streaming';
SET 'parallelism.default' = '4';

-- Output table with actual credentials
CREATE TEMPORARY TABLE IF NOT EXISTS user_features_scored (
  user_id                        BIGINT,
  amount                         DOUBLE,
  fraud_score                    DOUBLE
) WITH (
  'connector'                          = 'kafka',
  'topic'                              = 'fraud-scores-output',
  'properties.bootstrap.servers'       = 'kafka:9092',
  'properties.security.protocol'       = 'SASL_SSL',
  'properties.sasl.mechanism'          = 'PLAIN',
  'properties.sasl.jaas.config'        = 'org.apache.kafka.common.security.plain.PlainLoginModule required username="api-key-123" password="api-secret-456";',
  'format'                             = 'json'
);

-- Model Server Configuration
-- Server   : AWS SageMaker (sagemaker)
-- Endpoint : fraud-detection-endpoint-prod
-- Auth     : aws_sigv4
--   AWS SigV4 region: us-east-1
--   AWS service    : sagemaker
-- Timeout  : 5000ms  Retries : 2  Async parallelism : 4

CREATE TEMPORARY FUNCTION IF NOT EXISTS CALL_MODEL_UDF
  AS 'com.yourcompany.flink.udf.AWSageMakerAsyncUDF'
  LANGUAGE JAVA;

INSERT INTO user_features_scored
SELECT user_id, amount, CALL_MODEL_UDF(amount, tx_count_5m, avg_amount_5m,
  'fraud-detection-endpoint-prod', 'us-east-1') AS fraud_score
FROM user_features_out;

-- Real-time ML Inference Pipeline
-- Model Server  : Anthropic Claude
-- Auth          : x_api_key
-- Prediction    : risk_assessment (STRING)

CREATE TEMPORARY TABLE IF NOT EXISTS transactions_scored (
  user_id                        BIGINT,
  amount                         DOUBLE,
  risk_assessment                STRING
) WITH (
  'connector'  = 'jdbc',
  'url'        = 'jdbc:postgresql://localhost:5432/ml_results',
  'table-name' = 'public.fraud_assessments',
  'driver'     = 'org.postgresql.Driver',
  'username'   = 'flink_writer',
  'password'   = 'db-secret-pass'
);

-- Model Server Configuration
-- Server   : Anthropic Claude (anthropic)
-- Model    : claude-sonnet-4-6
-- Auth     : x_api_key
--   Header : x-api-key: sk-ant-api03-xxxx…[KEY]
-- Timeout  : 5000ms  Retries : 2

CREATE TEMPORARY FUNCTION IF NOT EXISTS CALL_MODEL_UDF
  AS 'com.yourcompany.flink.udf.AnthropicClaudeAsyncUDF'
  LANGUAGE JAVA;

INSERT INTO transactions_scored
SELECT user_id, amount,
  CALL_MODEL_UDF(CAST(amount AS STRING), 'claude-sonnet-4-6',
    'Classify this transaction as FRAUD or LEGITIMATE.', 128) AS risk_assessment
FROM transactions_raw;

Inference Pipeline Canvas

The Review step (Step 7) renders a fully interactive SVG inference pipeline canvas showing: source feature column nodes, an optional pre-process node, the model server hero node (with provider icon, async parallelism badge, and dashed orbit ring), an optional post-process node, and the output node. Passthrough columns are annotated below the output. The canvas supports the same zoom/pan/maximize controls as the Feature Engineering canvas.

History

Every completed inference pipeline is saved to history with source table, output table, model server, output alias, and the full generated SQL. Up to 20 entries. Each entry has an Insert SQL button to reload the pipeline into the editor without going through the wizard again.

Observability and Diagnostics v0.0.22

Six diagnostic tools accessible from the editor toolbar. Each opens as a modal with full interactive controls.

Live Pipeline Visualiser

Embedded in the SQL editor. Parses SQL in real time and renders a source-to-transform-to-sink DAG. Pan, zoom, and streaming animation while a job runs. Collapses to a vertical tab on the right edge (Ctrl+`). Recognises CREATE TABLE, CREATE VIEW, CREATE FUNCTION, INSERT INTO, CTEs, JOINs, GROUP BY, and UDF/VIEW names defined in the current script.

Schema Registry Browser

Connect to Confluent Schema Registry, Apicurio, Karapace, or AWS Glue SR. Browse all subjects with live search. Compare schema versions — breaking changes (removed required fields, unsafe type changes) shown in red, non-breaking in green. Validate a CREATE TABLE in the editor against the registered schema. One-click Generate CREATE TABLE with avro-confluent config and auto-detected watermark.

Backpressure Root Cause Analyser

Fetches per-vertex backpressure %, busy %, idle %, and records in/out per second. Identifies the highest-backpressure non-source vertex as the bottleneck. Traces the propagation chain. Correlates with checkpoint duration and GC pause times. Classifies root cause: slow sink, processing bottleneck, upstream starvation, GC pressure, or checkpoint slowness. Plain-English severity and per-finding actionable recommendations. SVG heatmap. Live auto-refresh every 8 seconds.

Watermark Health Monitor

Real-time per-vertex watermark lag, stall detection with configurable threshold (5s/10s/30s/60s), late-event rate, sparklines over ~4 minutes of history, and a cross-vertex timeline canvas showing all streams event-time positions relative to now.

Multi-Pipeline Dependency Graph

Fetches execution plans for all active jobs and builds a single cross-job topology: topics/tables as nodes, pipelines as nodes, data flow as directed edges. Detects dead-end topics, shared sources (multiple consumers), and circular dependencies. Animated particles on edges connected to RUNNING pipelines. Each pipeline node gets a unique palette color. Drag to reposition nodes. Double-click a node for detailed metrics.

SQL Profiler / Execution Replay

Flight recorder for running Flink jobs. Captures per-operator metric snapshots (records/s, backpressure %, busy %, bytes/s, checkpoint duration) at configurable intervals (2s to 30s) into a ring buffer of up to 360 snapshots. Replay any historical moment with a time scrubber — the DAG re-renders with backpressure tints and anomaly badges. Auto-play at 1x/2x/4x. Per-operator sparklines across the full recording. Export to JSON.

Reports

PDF reports generated from the Performance tab. Standard session report covers queries, timing, jobs, cluster info, and checkpoints. Admin Technical Report adds vertex-level metrics, JVM, and per-session breakdown. Admin Business/Management Report focuses on resource utilisation percentages and throughput summaries with plain-language annotations. Job Graph Report captures a single job's DAG topology, vertex states, record counts, and checkpoint history.

Admin Session

The Admin Session mode provides a privileged view of the entire Flink cluster across all users and sessions. Select Admin mode on the connect screen and enter the admin passcode.

Connector JARs

./scripts/download-connectors.sh
# Override: ./scripts/download-connectors.sh --flink-version 1.19.1

.\scripts\download-connectors.ps1
# Override: .\scripts\download-connectors.ps1 -FlinkVersion 1.19.1

FROM flink:1.19.1-scala_2.12-java11
USER root
RUN apt-get update && apt-get install -y libgomp1 && rm -rf /var/lib/apt/lists/*
COPY ./connectors/*.jar /opt/flink/lib/
USER flink

Str:::lab Studio Kube Operator

Manages Studio deployments as first-class Kubernetes CRDs. Deploy and lifecycle-manage Studio instances with a single kubectl apply. Built with Kopf (Python). Repository: github.com/coded-streams/strlabstudio-operator

CRD Reference — StrlabStudio

Short name: kubectl get fss  ·  Group: codedstreams.io/v1alpha1

helm repo add strlabstudio https://coded-streams.github.io/strlabstudio-operator/charts
helm repo update
helm install strlabstudio-operator strlabstudio/strlabstudio-operator \
  --namespace flinksql-system --create-namespace

apiVersion: codedstreams.io/v1alpha1
kind: StrlabStudio
metadata:
  name: dev-studio
  namespace: flink
spec:
  gateway:
    host: flink-sql-gateway
    port: 8083
  jobmanager:
    host: flink-jobmanager
    port: 8081

apiVersion: codedstreams.io/v1alpha1
kind: StrlabStudio
metadata:
  name: prod-studio
  namespace: flink
spec:
  image: codedstreams/strlabstudio:v0.0.23
  replicas: 2
  gateway:
    host: flink-sql-gateway
    port: 8083
  jobmanager:
    host: flink-jobmanager
    port: 8081
  service:
    type: LoadBalancer
    port: 80
  resources:
    requests: {cpu: "100m", memory: "128Mi"}
    limits:   {cpu: "500m", memory: "256Mi"}

Deploy Options

docker compose up -d

python3 -m http.server 8080

Flink Compatibility

Changelog