IREX StreamVLM™ Brings Prompt-Defined Real-Time Video Analytics to Public Safety

MURRIETA, CA — IREX (IrexAI Inc.), developer of the IREX Ethical AI Platform for public safety, today announced StreamVLM™, the world’s first Vision-Language Model (VLM) analytics for public safety. StreamVLM™ ships in IREX 4.43, scheduled for release between December 2026 and early January 2027. StreamVLM™ lets public safety, transportation, and critical infrastructure teams create custom real-time video detectors from plain-English text prompts. There is no dataset to collect, no image to annotate, and no neural network to retrain: a new detector starts as a sentence and can be running in minutes, not months.

What sets StreamVLM™ apart from general-purpose AI assistants such as ChatGPT or Gemini is hardware efficiency and scale. Its architecture and models are purpose-built for live video, so a single GPU serves far more cameras than a general-purpose model could, and the IREX platform orchestrates hundreds of StreamVLM™ scenarios across thousands of cameras. StreamVLM™ also runs on premises, inside the customer’s own infrastructure: no video and no data leaves the country or the private network.

The Problem StreamVLM™ Solves

Every new hazard has traditionally meant a new model. An agency that wanted to detect something its analytics did not already cover had to gather thousands of images, label them, train a detector, and roll it out across its servers, a cycle that takes months for each request.

Large Vision-Language Models remove the training step, because they understand what they see in words. They are also slow: a single VLM call takes roughly 1 to 2 seconds per frame on a GPU, while a production camera network needs tens of frames per second per GPU across many streams. Pointing a VLM at every frame of every camera is not practical.

StreamVLM™ resolves that tension with a two-stage funnel that spends GPU time only where it matters, reaching 97.9–100.0% precision across its eight production-approved detection segments.

How the Two-Stage Funnel Works

On a real camera network, critical incidents are rare: the overwhelming majority of frames show routine activity. StreamVLM™ puts a fast multimodal filter in front of a careful VLM verifier, so heavy GPU compute is spent only when something is actually happening.

1. Stage 1: Fast Multimodal Embedder (semantic margin filter). The platform samples keyframes from the live stream, or from a polygon Region of Interest (ROI) inside it, and projects each frame into a shared vision-language embedding space. It compares the frame with cached POSITIVE target phrases and NEGATIVE false-alarm phrases and computes the similarity margin in milliseconds. Routine frames and predictable look-alikes stop here.
2. Stage 2: Vision-Language Model Verification (CHECK). Only when the Stage 1 score crosses the configured trigger (for example 80%) and holds for the required duration does StreamVLM™ call the VLM. Guided by the detector’s CHECK instruction, the VLM inspects the scene and returns a structured JSON verdict (yes or no), a confidence score, and a plain-English description of the evidence it saw.

The effect is easy to measure. In a 5-minute industrial courtyard video sampled at 1 FPS (336 frame checks), workers produced sparks near a metal brazier for four minutes. With “welding sparks at a construction site” set as a negative anchor, those sparks scored 55–76%, below the 80% trigger. Only 29 of the 336 frames crossed the trigger, and they coalesced into just 5 Stage 2 VLM calls, a reduction of more than 91% in VLM invocations. When real flames broke out, StreamVLM™ confirmed all three fire episodes with 95% confidence.

Three-Part Prompts: False Alarms Designed Out

Text-to-image embedding models handle negation poorly. A prompt for “no smoke” still matches the word “smoke”, and a positive-only prompt for “camels on the road” can fire on a passing horse, because both animals sit closer to each other in embedding space than either does to empty asphalt.

So StreamVLM™ does not rely on a single sentence. Administrators build each detector in the system-wide VLM Detector Catalog from three parts:

• POSITIVE (1–16 target phrases): short visual descriptions of the event across lighting conditions and camera angles.
• NEGATIVE (0–16 confounder phrases): known non-events and optical look-alikes that suppress false triggers at Stage 1.
• CHECK (Stage 2 verification rule): explicit visual criteria the VLM uses to confirm the incident and explain its decision.

Example 1: Chimney and Industrial Smoke

POSITIVE:
thick black smoke rising from a chimney
dark gray smoke billowing from a rooftop chimney
a plume of smoke coming out of a house chimney
white smoke rising from a residential chimney
a smoking chimney over houses in winter

NEGATIVE:
white steam evaporating from a vent
fog over rooftops in the morning
low clouds over a residential area
a dirty camera lens with smudges
lens flare and sun glare in the sky
a dark tree silhouette against the sky
clear sky over houses with no smoke

CHECK:
Determine whether smoke is actively being emitted from a chimney or flue.
A real chimney smoke plume originates at a chimney mouth, is attached to it,
widens and fades as it rises, and drifts or changes shape across frames.
Distinguish steam, fog or low clouds, lens dirt, glare, and dark static objects.
If smoke is present, classify its color as white, dark, or unknown.

Example 2: Open Fire vs. Welding Sparks

To catch unauthorized burning or an early fire on an industrial site without alarming on routine maintenance, operators pair positive fire anchors (“open flames burning”, “a fire with thick smoke”, “a burning car on fire”) with negative confounders such as sun glare, vehicle headlights, glowing light bulbs, and “welding sparks at a construction site”. Stage 1 keeps the sparks below the trigger; Stage 2 confirms the genuine fire and writes evidence an operator can read at a glance: “Visible smoke plume rising from a small fire in a metal container, with glowing embers and burning material at the base.”

The same second look pays off on city streets. In multi-day live trials on public street cameras, watching for fallen pedestrians, emergency vehicles, and scooters or bicycles on sidewalks, street promoters in full-body bear costumes repeatedly tripped the Stage 1 shape similarity for a person lying down. Each time, the Stage 2 VLM looked again and rejected the alarm.

Production-Approved Performance

Every StreamVLM™ detector profile is validated before it is approved. The release gate is strict: a detector ships only when it reaches at least 97.0% precision and at least 90.0% recall on held-out test data.

Measured accuracy for the eight production-approved detection segments, with the expected throughput on one NVIDIA L40S, the data-center GPU IREX runs in its Dallas facility for US customers:

Detection Segment	Precision	Recall	F1 Score	Expected Throughput (1× NVIDIA L40S)
Weapon in Frame (Weapon Detection)	98.9%	95.8%	0.973	~3 FPS
Smoke & Fire (Fire & Smoke)	99.2%	96.1%	0.976	~11 FPS
Any Smoke (Fire, Chimney, Exhaust)	98.2%	94.7%	0.964	~51 FPS
Hazardous Fire Smoke (Fire vs. Steam/Exhaust)	97.9%	92.0%	0.949	~85 FPS
Road Works & Construction Zones	97.9%	97.9%	0.979	~40 FPS
Unauthorized Street Vending	100.0%	97.1%	0.985	~77 FPS
Camera Sabotage: Defocus	100.0%	94.6%	0.972	~15 FPS
Wildlife / Camels on Roadway	100.0%	100.0%	1.000	~5 FPS

Precision, recall and F1 are measured test-set results on the IREX validation benchmark. Throughput is an engineering estimate for one NVIDIA L40S, scaled from benchmark runs by GPU core count; a standard IREX GPU node carries two. Real-world accuracy depends on camera placement, lighting, and Region of Interest configuration.

The Right Frame Rate for Each Task

Not every application needs to look at every second of video. StreamVLM™ checks each camera at the interval the task calls for, and that interval decides how many cameras one GPU can cover:

• Time-critical safety: for fire and smoke detection, IREX recommends checking a frame every 10 to 30 seconds for a fast response.
• Slow-changing conditions: for camera lens monitoring or street-cleaning control, StreamVLM™ can review a frame every 10 to 60 minutes.

Because throughput is shared across cameras, a longer interval multiplies coverage. At one frame every 30 seconds, a detector running at ~85 FPS could in principle cover more than 2,000 camera channels on a single L40S, and at intervals measured in minutes the same GPU reaches far further. That is what lets StreamVLM™ scale across the thousands of cameras of a city or a state. Final sizing for each deployment is done by IREX engineering.

Built for Operational Control and Ethical AI Governance

StreamVLM™ plugs into existing IREX command-center workflows:

• Single-frame detector testing: before a detector goes live, administrators test the prompt against a live camera snapshot or an uploaded JPEG or PNG image and see the exact similarity score, the VLM verdict, and its explanation.
• Multiple detectors per camera: several StreamVLM™ detectors can run on one camera channel, each with its own polygon ROI. When detectors share an ROI, Stage 1 computes the frame embedding once and reuses it for every active rule. The per-camera ceiling depends on the customer’s hardware and is confirmed during the pilot.
• Fail-open by design: if the VLM verification service is briefly unavailable during a load burst, StreamVLM™ still raises the Stage 1 alert and marks it “Additional check not performed”, so an urgent safety incident is never silently dropped.
• Transparency by Design and accountability: every StreamVLM™ alert carries the full frame, the ROI crop, the similarity percentage, and the VLM’s plain-text evidence. Searching events requires a Case ID, and every configuration change and search is recorded in the immutable Audit Logbook under dedicated Role-Based Access Control (RBAC) permissions.

Availability and Roadmap

StreamVLM™ real-time alerting ships in IREX 4.43, scheduled for release between December 2026 and early January 2027. It runs on a GPU node inside the customer’s IREX deployment, sized by IREX engineering, while the rest of the platform keeps running on standard servers. Descriptor-based retrospective natural-language search across stored video archives is on the 2027 IREX product roadmap.

Explore StreamVLM™ →

About IREX

IREX is an Ethical AI Platform for public safety, smart cities, and traffic management, with 250,000+ cameras deployed globally across 8 countries and 100,000+ cameras proven in a single instance. Built on the principle of Transparency by Design, IREX turns existing camera networks into intelligent, auditable tools while customers retain full sovereignty over their data.