Video Playback

A video in PlasmaEngine2 is a texture that changes every frame. You import a video file as a Video asset. The asset is a 2D texture resource, so anything that takes a texture takes the video: a material slot, a sprite, a GameUI image, a light cookie or the boot splash. A plVideoPlayerComponent (or the boot flow, or a console command) starts the player behind the asset. From then on, the decoder thread fills frames and the render thread uploads the frame that is due at the clock.

Layer	Type	Responsibility
Asset	plVideoAsset (editor) → .plBinVideo	Probes the file, stores a poster frame and the original container bytes unchanged.
Resource	plVideoTextureResource	A plTexture2DResource selected automatically for .plBinVideo. Shows the poster until something plays.
Player	plVideoPlayer	Owns the decoder, the frame ring and the clock. One per asset, shared by every user of it.
Backend	plVideoDecoder	Demuxes and decodes one file. Windows ships the Media Foundation backend, and others register through plVideoDecoderFactory.
Component	plVideoPlayerComponent	Play, pause, seek, loop and volume controls, blackboard outputs, and average colour sent to a light.

Supported files

The engine keeps the container as it is and asks the platform to decode it, so support depends on the machine. The Windows backend uses Media Foundation, and H.264 in MP4 with AAC audio is the baseline format. HEVC, VP9, AV1 and Opus require suitable installed decoders. Check the asset transform log for the backend and codecs that accepted the file, and verify playback on your target machines.

warning

Shipping profiles

Use MP4 with H.264 and AAC for a Windows-only title. A cross-platform title needs AV1 with Opus and the plasma.av1 package, which is not available yet. Until it is, non-Windows builds show the poster frame.

Setting up a Video asset

1. Drop the video file (.mp4, .mov, .m4v, .webm, .mkv) into the asset browser, or create a Video asset and pick the file in its File property. Dropping copies the file into the project.
2. Set Poster Time to the moment that should show before playback (the default is the first frame).
3. Leave Decoder on Auto unless a specific backend must be forced.
4. Turn Audio Track off for videos that must never make sound (they then run on the wall clock).
5. Turn Generate Mips off only for videos that are used in the UI alone. Everything in the world, and every light, needs the mip chain.
6. Transform. The output is AssetCache/…/Name.plBinVideo: a small header, the poster PNG and the untouched video bytes. Raw video files are excluded from exports, and the bake is what ships.

The asset window plays the video through the same player the game uses, with the usual transport (pause, restart, loop, speed) and a scrubber.

Asset playback and stream settings

Property	Default	Purpose
Auto Play	false	Starts playback when the texture loads, including when used directly by a material or light cookie without a player component.
Loop	false	Loops asset-driven playback. A player component can inherit or override this setting.
Video Stream	0	Selects the video stream for containers with multiple streams.
Audio Stream	0	Selects the audio stream when Audio Track is enabled.

Use asset Auto Play for an always-running texture. Use a player component when scripts need to control playback, publish blackboard values or receive end events.

Playing a video in a scene

1. Give a mesh a material whose base texture is the Video asset (a Fullbright material for a screen), or let the component do it: add a plVideoPlayerComponent to the object and set Screen Material Slot to the mesh slot that should show the picture.
2. Set Video to the asset. The component's Auto Play starts it when the simulation starts. Set Loop to Use Asset, Loop or Once, then set Playback Rate, Volume, Muted and Bus as needed.
3. Scripts call Play, Pause, Stop, Seek, SetLooping, SetPlaybackRate, SetVolume, IsPlaying, GetPosition, GetDuration and GetAverageColor. A plMsgSetPlaying message toggles playback like any other playable component.
4. When the clip ends (and on every wrap while looping) the component sends plMsgVideoReachedEnd through its event sender.

Two components on the same asset share one player and one texture. Stopping one stops the picture for both. Use two assets for two independent screens showing the same file.

Playback sharing diagram. Each consumer of the same asset sees the same playback position.

Blackboard outputs

Progress Entry receives 0 to 1, Playing Entry the playing flag, Average Color Entry the frame's mean colour. The component looks for a plBlackboardComponent on its object hierarchy first and falls back to the global blackboard named in Blackboard. GameUI widgets bind to these entries the same way they bind to any other value.

Video as a light source

There are two ways for a screen to light the room, and a screen should use one of them, not both.

Analytic: a light wearing the video

1. Place a plSpotLightComponent or plRectLightComponent in front of the screen, facing the room.
2. Set its Cookie to the Video asset. The light's pattern is now the picture, updated every frame, in raster, ReSTIR direct lighting, probe direct lighting and the path tracer alike.
3. Brightness comes from Intensity, and the cookie only modulates it. Enable Auto Play on the Video asset, or use a plVideoPlayerComponent to start it. Assigning the cookie alone does not start an asset with Auto Play disabled.
4. On the screen's own material turn Emissive Lighting off, so the emissive picture is visible but does not light the room a second time.

Emissive: the screen is the light

Set the material's emissive texture to the Video asset and leave Emissive Lighting on. Voxel GI, ray-traced GI and ReSTIR emissive triangle lights then carry the picture into the room with soft, area-correct light. With Rendering.ReSTIRDI.TexturedEmissive on (the default), the ReSTIR light samples the picture at the sampled point rather than a flat colour, and its importance follows the frame's mean.

A lamp beside the TV

Set Apply Average Color To on the video component to any object whose components handle plMsgSetColor (lights, meshes). Every frame the component sends the frame's mean colour, scaled by Average Color Scale, applied with Average Color Mode (RGB by default, so the light's intensity stays yours).

Boot video

In Project Settings → Startup a splash screen entry can carry a Video instead of an image. A video entry with Duration 0 plays to the end of the clip, and a positive duration cuts it. Audio plays the clip's audio on the master bus, Loop keeps it playing while the entry is held for loading. Skippable with Min Duration works as for images. The video can decode before project plugins are loaded.

Console and diagnostics

VideoPlay("Videos/Intro.mp4")   -- or a Video asset guid "{ … }"; a bare file plays on the sprite-less path
VideoPause()  VideoSeek(3.5)  VideoStop()
Video.Stats = true               -- per-player stats: decoded, presented, dropped, uploaded MB, clock kind

From the command line, -exec "@2000; VideoPlay(\"…\"); @4000; VideoStop(); quit" runs a scripted session, and @N waits N milliseconds.

Property reference: plVideoPlayerComponent

Property	Default	Description
Video	None	The Video asset.
AutoPlay	true	Start when the simulation starts.
Loop	UseAsset	Use the asset's Loop setting, force Loop, or force Once. SetLooping(bool) selects Loop or Once. plMsgVideoReachedEnd fires on every wrap.
PlaybackRate	1.0	0.05 to 8. Audio pitch follows.
Volume	1.0	Linear, 0 to 2.
Muted	false	Silences without changing Volume.
Bus	(master)	Mixer bus for the audio stream.
ScreenMaterialSlot	-1	Mesh slot that receives an unlit material showing the video. -1 leaves materials alone.
Blackboard	None	Global blackboard name, used when no blackboard component is on the hierarchy.
ProgressEntry, PlayingEntry, AverageColorEntry	None	Blackboard entries written every frame.
ApplyAverageColorTo	None	Object that receives the frame's mean colour as plMsgSetColor.
AverageColorScale	1.0	Multiplier on the colour sent.
AverageColorMode	SetRGB	How the colour is applied at the receiver.

Settings

CVar	Default	Description
Video.PlanarUpload	true	Ask decoders for NV12 and convert on the GPU (1.5 bytes per pixel uploaded instead of 4).
Video.GenerateMips	true	Mip chain for textures the console player creates. Assets carry their own setting.
Video.QueueDepth	4	Decoded frames buffered per player.
Video.UploadBudgetMB	64	Bytes uploaded per rendered frame across players. Every playing video still gets at least one frame.
Video.DecodeThreads	1	Decode threads, with players distributed round-robin. Startup only.
Video.AudioClock	true	Uses the audio device as master clock when the clip has sound. Off forces the wall clock.
Video.Stats	false	Publish per-player statistics.
Video.LogProbe	false	Log which backends were tried for a file and why they declined.
Video.ConvertPlanes	true	Diagnostic. Off uploads planes but skips the conversion pass.
Boot.VideoAudio	true	Play the boot video's audio track.
Rendering.ReSTIRDI.TexturedEmissive	true	ReSTIR emissive triangle lights sample the emissive texture at the sampled point.

Troubleshooting

Symptom	Cause	Fix
Poster shows, nothing plays	Asset Auto Play is off and no component or console command started the player, or no backend accepts the file on this machine.	Enable asset Auto Play or add a plVideoPlayerComponent. Check the log line Video: opened … or the probe log with Video.LogProbe.
Picture but no sound	No audio middleware is loaded (the MiniAudio package is missing from the project), Audio Track is off, or Muted is on.	Stats show clock wall when there is no audio path. Add the audio package to the project.
Video plays at the wrong size or with a green band	The bake was transformed from a different file than the one now on disk.	Re-transform the asset. The log names the two sizes.
Light cookie shows a still image	The light's cookie is a Video asset but nothing plays it.	Enable Auto Play on the asset or add a player component for that asset.
The room is lit twice	Screen material has Emissive Lighting on and a rect light wears the same video.	Turn Emissive Lighting off on the screen material.
Frames dropped on a 4K clip	Upload budget or decode too slow for the frame rate.	Keep Video.PlanarUpload on, raise Video.UploadBudgetMB and use a hardware-decodable codec.

note

How the clock works

With sound, the audio device is the master: the player counts the PCM frames the mixer has rendered and shows the picture that matches. Without sound the wall clock drives playback. Either way frames older than the clock are dropped, never shown late.