wityAudioProfile — Audio Loudness Analysis

Lambda service that measures actual audio loudness over time from a wity-scene document. Downloads all ws-video and ws-audio media, decodes their audio tracks via FFmpeg, computes windowed RMS loudness, applies XML volume multipliers, and returns per-element and power-mixed loudness profiles in dBFS.

Unlike the rendering pipeline, this is a standalone analysis service — not routed through witySceneRender. Call wityAudioProfile directly.

Node.js / Lambda only. Requires FFmpeg.

---

analyzeAudioProfile(sceneXml, options)

import { analyzeAudioProfile } from './index.js';  // internal service

analyzeAudioProfile(
  sceneXml: string,
  options?: {
    windowMs?:   number,   // RMS window size in ms (default 100)
    sampleRate?: number,   // decode sample rate Hz (default 16000)
  }
): Promise<AudioProfileResult>

Option	Type	Default	Description
windowMs	number	100	RMS measurement window size in milliseconds
sampleRate	number	16000	PCM decode sample rate. Lower = faster, less resolution

---

Output shape

interface AudioProfileResult {
  sceneDuration: number;          // resolved scene duration in seconds
  windowMs:      number;          // window size used
  sampleRate:    number;          // sample rate used
  elements:      ElementProfile[];
  mixed:         LoudnessProfile; // power-sum mix of all element profiles
}

interface ElementProfile {
  id:       string;
  tag:      'ws-video' | 'ws-audio';
  src:      string;
  begin:    number;               // scene timeline offset (seconds)
  dur:      number | null;        // null = resolved from source duration
  volume:   number;               // XML volume attribute (0–1)
  hasAudio: boolean;              // false if file has no audio track or muted="true"
  profile:  LoudnessProfile | null;  // null if hasAudio is false or decode failed
}

interface LoudnessProfile {
  startTime:   number;     // scene timeline offset of window[0] (seconds)
  windowCount: number;     // number of windows
  windowMs:    number;     // window size in ms
  rmsLinear:   number[];   // per-window RMS amplitude 0.0–1.0, volume-adjusted
  rmsDbfs:     number[];   // per-window dBFS (20 * log10(rmsLinear)); -Infinity for silence
  peakDbfs:    number;     // loudest window in dBFS
  avgDbfs:     number;     // RMS average in dBFS
}

Volume adjustment: rmsLinear values have the element's XML volume multiplier applied before dBFS conversion — what you see is what you'd hear in the final mix.

Mixed profile: Power-additive (sum of rms² → sqrt) across all elements, mapped to the scene timeline. startTime is always 0.

---

What is processed

Element type	Processed?
ws-video	Yes — audio track decoded; skipped if muted="true" or no audio track
ws-audio	Yes — audio decoded
ws-rect, ws-text, ws-image	No — silently ignored

---

Lambda deployment

Function name: wityAudioProfile

Input

{
  "sceneXml": "<wity-scene>...</wity-scene>",
  "options": {
    "windowMs":   100,
    "sampleRate": 16000
  }
}

Response

The full AudioProfileResult object:

{
  "sceneDuration": 15.0,
  "windowMs": 100,
  "sampleRate": 16000,
  "elements": [
    {
      "id": "v1", "tag": "ws-video", "src": "https://...", "begin": 0, "dur": 15,
      "volume": 0.8, "hasAudio": true,
      "profile": {
        "startTime": 0, "windowCount": 150, "windowMs": 100,
        "rmsLinear": [0.12, 0.15, "..."],
        "rmsDbfs": [-18.4, -16.5, "..."],
        "peakDbfs": -6.2, "avgDbfs": -18.1
      }
    }
  ],
  "mixed": {
    "startTime": 0, "windowCount": 150, "windowMs": 100,
    "rmsLinear": ["..."], "rmsDbfs": ["..."],
    "peakDbfs": -5.8, "avgDbfs": -16.3
  }
}

Lambda config

Setting	Value
Function name	wityAudioProfile
Memory	1024 MB
Timeout	120 s
Ephemeral storage	2048 MB
Runtime	Node.js 20
FFmpeg layer	arn:aws:lambda:ap-south-1:175033217214:layer:ffmpeg:1

---

Edge cases

Scenario	Behaviour
No audio track in video file	hasAudio: false, profile: null
muted="true" on ws-video	Treated as no audio
Infinite dur	Source duration probed via ffprobe; clamped to sceneDur - begin
trimOut set	FFmpeg -to trimOut limits decode window
Empty scene (no ws-video/ws-audio)	Returns empty elements[] and a zero mixed profile
Decode error on one element	Element included with hasAudio: true, profile: null; error logged; other elements continue