Rendering video¶

VideoView shows a video track on screen: your own camera, a participant's video, a screen share or a track you fill with your own frames. It is a UIView, so you add it to your layout like any other view and give it a track.

VideoView is available on iOS only. There is no macOS version.

This page covers how to attach a track, how the picture fills the view, mirroring and rotation, the render modes, pausing and freezing, the rendering state and its events, what the view shows while your camera is muted, pinch to zoom, the debug overlay, custom renderers and SwiftUI.

Show a track¶

Set VideoView.track to the track you want to show. The view starts drawing with the next frame; set it to nil to stop and clear the view.

Swift

import RevueSDK

/// Show a participant's camera in their tile, or the App's placeholder when the camera is muted.
/// firstCameraVideoTrack still returns the track while the participant is muted, so check isMuted.
/// Call this again from didUpdateIsMuted.
func showCamera(of friendId: String, in videoView: VideoView) -> Bool {
    let camera = Room.shared.remoteParticipants[friendId]?.firstCameraVideoTrack
    guard let camera, !camera.isMuted else {
        videoView.track = nil
        return false  // show the placeholder
    }
    videoView.track = camera
    return true
}

The view does not keep its track¶

VideoView.track is a weak reference: the view never keeps a track alive. Who keeps it depends on where the track comes from:

Track	Kept by
A remote participant's track	Its publication, while the participant publishes it.
`LocalParticipant.cameraTrack`, `microphoneTrack`, `screenShareTrack`	`LocalParticipant`, until you call `destroy()`.
A track from `LocalVideoTrack.createBufferTrack(name:source:)`	Your App. Keep it in a property for as long as you show it.

When the last owner lets a track go, for example when a participant leaves, the track is released, VideoView.track becomes nil and the view clears. A track you created yourself and did not store is released straight away, and the view stays blank.

Set VideoView.track to nil when the view is no longer shown, for example when a collection view cell is reused, so that the view stops drawing a track nobody sees.

Swift

import RevueSDK
import UIKit

/// Preview a track the App creates itself. VideoView holds its track weakly: keep the track in a
/// property, or it is released and the view goes blank. Clear the view when it is no longer shown.
@MainActor
final class OwnTrackPreview {
    let track = LocalVideoTrack.createBufferTrack(name: "ipcam", source: .camera)
    let videoView = VideoView(frame: .zero)

    func show() {
        videoView.track = track
    }

    func hide() {
        videoView.track = nil
    }
}

Fill the view¶

VideoView.layoutMode decides how a picture whose shape differs from the view's is placed:

`VideoView.LayoutMode`	Result
`.fill` (default)	Fills the whole view and crops what sticks out. No black bars.
`.fit`	Shows the whole picture, with black bars where it does not cover the view.

The view's background is black, so bars and an empty view are black.

Mirror and rotate¶

VideoView.mirrorMode mirrors the picture on this screen only. The participants you send to do not see the mirroring; to mirror what you send, use CameraCaptureOptions.isMirrored (see Camera and capture options).

`VideoView.MirrorMode`	Result
`.auto` (default)	Mirrors your own front camera, as people expect from a selfie preview. Your back camera, remote tracks, screen shares and buffer tracks are not mirrored. When you switch cameras, the view follows.
`.off`	Never mirrors.
`.mirror`	Always mirrors.

VideoView.rotationOverride turns the picture clockwise by a VideoRotation (._90, ._180 or ._270). Frames reach the view already upright: the sender turns the pixels upright before sending (see Your own frames and audio). rotationOverride is an extra angle your App chooses for display, for example to match a screen mounted on its side. nil, the default, and ._0 show the picture as it is.

layoutMode works on the rotated picture: a 1280×720 picture turned by ._90 fills or fits the view as a 720×1280 one.
Rotation and mirroring add up: the picture is turned first, then mirrored.
With the Metal renderer the new angle applies from the next frame; with the sample buffer renderer it applies as soon as you set it. See Render modes.
You can set it while frames arrive.

Render modes¶

VideoView.renderMode picks how the view draws:

`VideoView.RenderMode`	Draws with
`.auto` (default)	Metal when the device supports it, otherwise an `AVSampleBufferDisplayLayer`.
`.metal`	Metal. Falls back to the sample buffer layer when Metal is not available.
`.sampleBuffer`	An `AVSampleBufferDisplayLayer`.

VideoView.isMetalAvailable() tells whether Metal can be used. Changing the mode clears the view; the next frame draws with the new mode.

VideoView.avSampleBufferDisplayLayer gives the sample buffer layer, for example to show the video in picture in picture, which accepts this kind of layer only. Set renderMode to .sampleBuffer first: in Metal mode the property is nil.

Pause, hide and freeze¶

Three properties stop the view from drawing new frames. None of them affects the track: frames keep arriving, and the participants you send to see no difference.

Property	While set	The view shows
`isEnabled = false`	New frames are dropped.	Nothing: the view is cleared.
`isHidden = true`	New frames are dropped.	Nothing: the view is hidden and cleared.
`isFrozen = true`	New frames are dropped.	The last frame, kept on screen.

isEnabled suits a view that still exists but cannot be seen, such as a reused cell. When isEnabled goes back to true or the view is shown again, it draws from the next frame on.

A frozen view counts as not rendering after 2 seconds (see below). isFrozen belongs to the view, not to the track: when the track changes or the view is cleared, the frozen picture goes away but isFrozen stays true, so the view stays blank until you set it back to false.

Rendering state and events¶

Two properties tell whether video is showing:

VideoView.isRendering is true while frames are being drawn. It turns true with the first frame and false when no new frame has been drawn for 2 seconds, when the view is cleared, or when the track changes.
VideoView.didRenderFirstFrame turns true with the first frame drawn and stays true until the track changes.

A track can exist before its first frame arrives, for example while a remote participant's video is still connecting. Use isRendering to decide when to show a placeholder over the view.

A VideoViewDelegate hears about two changes:

Event	When
`videoView(_:didUpdate:)` with a `Bool`	`isRendering` changed.
`videoView(_:didUpdate:)` with a `CGSize`	The view's size changed, in points. Not sent when the view only moves, nor for its first layout when the size equals the frame you created it with.

Add a delegate with VideoView.add(delegate:) and remove it with VideoView.remove(delegate:) or VideoView.removeAllDelegates(); VideoView.delegates is the same list. Delegates are held weakly. Like every delegate in RevueSDK, both events arrive on the main thread, even when the frame was drawn on another thread.

Swift

import RevueSDK
import UIKit

/// Cover a tile with a placeholder until its video shows, and follow the tile's size.
/// Both events arrive on the main thread.
@MainActor
final class TileController: @MainActor VideoViewDelegate {
    let videoView = VideoView(frame: .zero)
    let placeholder = UIView()

    init() {
        videoView.add(delegate: self)  // held weakly
    }

    func videoView(_ videoView: VideoView, didUpdate isRendering: Bool) {
        placeholder.isHidden = isRendering
    }

    func videoView(_ videoView: VideoView, didUpdate size: CGSize) {
        // The new size of the view in points; not sent when it only moves.
        print("tile is now \(size.width)×\(size.height)")
    }

    func stopWatching() {
        videoView.remove(delegate: self)
    }
}

When your camera is muted¶

For your own camera, the view shows LocalVideoTrack.mutePlaceholder while the track is muted or the camera is off, the same picture the participants you send to receive.

With a mutePlaceholder set, the view shows it, and changing it while muted shows the new one at once.
With no mutePlaceholder, the view is cleared: it does not keep the last camera frame. Setting mutePlaceholder to nil while muted clears the view.
When you unmute or turn the camera on, camera frames replace the placeholder.

A view you attach while the camera is already muted or off starts with the placeholder, or empty.

A custom renderer is not told to clear. It receives the placeholder through render(pixelBuffer:) like any frame, but when there is no placeholder it gets nothing and keeps whatever it drew last; watch the track's mute state yourself to clear it.

For a remote track, the view clears when the track becomes muted or stops.

Pinch to zoom¶

On your own camera, the user can pinch the view to zoom. Pinching changes the camera's own zoom (CameraCapturer.device), not the picture on screen, so the participants you send to see the zoomed picture too. On a remote participant's video, a screen share or a buffer track, pinching does nothing.

Pinch to zoom is off by default. Turn it on with VideoView.isPinchToZoomEnabled or VideoView.pinchToZoomOptions, a VideoView.PinchToZoomOptions:

Option	Effect
`.zoomIn`	The user can zoom in beyond the camera's default zoom, up to the camera's maximum.
`.zoomOut`	The user can zoom out below the default zoom, down to the camera's minimum.
`.resetOnRelease`	When the user lifts their fingers, the zoom goes back to the default.

Without .zoomIn and .zoomOut pinching does nothing. Without .zoomOut the zoom never goes below the default, and without .zoomIn never above it.

isPinchToZoomEnabled = true adds .zoomIn only: the user can zoom in, not out. Setting it to false removes .zoomIn only; reading it returns true while either .zoomIn or .zoomOut is set.
VideoView.isAutoZoomResetEnabled adds or removes .resetOnRelease.
The default zoom is 2 on cameras made of several lenses whose 1× is the ultra wide lens (triple and dual wide cameras), so that 2× matches the wide lens. It is 1 on other cameras, kept within what the camera supports.
When you change the options, the camera's current zoom is brought back within the new range, or to the default when .resetOnRelease is set.

Debug overlay¶

Set VideoView.isDebugMode to true to draw an overlay on the view, for tracking down video problems. It shows:

Line	Meaning
sid	The track's sid, or nil without a track.
resolution	Width × height of the last frame drawn, or `-` before the first frame.
firstFrame	`didRenderFirstFrame`.
isRendering	`isRendering`.
renderMode	The `renderMode` you set, and the renderer actually used (`metal` or `sampleBuffer`).
renderers	How many renderers the track has, this view included.
FPS	Frames drawn in the last second, updated every second.

The overlay's border is green while the view is rendering and red when it is not. The overlay does not take touches. Setting isDebugMode back to false removes it.

Adaptive stream¶

Four members belong together: VideoView.isAdaptiveStreamEnabled, VideoView.adaptiveStreamSize, VideoView.adaptiveStreamPixelDensity and VideoView.set(size:). Together they let the participant whose video you show send it at the size you display it, instead of always at their capture size.

尚未支援

自動調整解析度目前沒有作用：不論 VideoView 多大，對方都照原本擷取的尺寸送。 isAdaptiveStreamEnabled 目前不能設定，set(size:) 也沒有作用。

The view tells the participant it shows how big it is. The size is adaptiveStreamSize: the view's size times the pixel density, in pixels.
adaptiveStreamPixelDensity is the density: .auto (default) uses the screen's scale, .fixed(_:) a value of your own. Either is capped at VideoView.AdaptiveStreamPixelDensity.maxDensity (3), so a large screen does not ask for more than it needs.
A view whose isAdaptiveStreamEnabled is false is not counted. A view that has not been laid out yet, is hidden or has isEnabled false always reads false.
The view follows its own size from layout, so you do not need to call set(size:).

Custom renderers¶

To get decoded frames yourself, for example to draw them with your own engine, implement VideoRenderer and add it to a track with VideoTrack.add(videoRenderer:); remove it with VideoTrack.remove(videoRenderer:). The track holds renderers weakly, so keep yours. VideoView is a VideoRenderer itself, and the track adds it when you set VideoView.track.

VideoRenderer has one method, render(pixelBuffer:). The track calls it once per frame; your App never calls it on a VideoView. It is called on the thread that delivers the frame, not on the main thread: the camera's capture queue for your camera, the thread your App pushes buffers from for a buffer track, and the receiving thread for a remote track. Return quickly and move work to the main thread yourself when you touch UI.

A custom renderer receives frames only. It is not told when the view would be cleared; see When your camera is muted.

SwiftUI¶

RevueSDK has no SwiftUI view. Wrap VideoView in a UIViewRepresentable:

Swift

import RevueSDK
import SwiftUI

/// Show a video track in SwiftUI. RevueSDK has no SwiftUI view of its own: wrap VideoView in a
/// UIViewRepresentable. The view does not keep the track alive, so whoever passes it in must hold it.
struct TrackVideoView: UIViewRepresentable {
    let track: VideoTrack?
    var layoutMode: VideoView.LayoutMode = .fill
    var mirrorMode: VideoView.MirrorMode = .auto

    func makeUIView(context: Context) -> VideoView {
        VideoView(frame: .zero)
    }

    func updateUIView(_ view: VideoView, context: Context) {
        view.layoutMode = layoutMode
        view.mirrorMode = mirrorMode
        // Setting the same track again restarts it (first frame, rendering state), so only set a new one.
        if view.track !== track {
            view.track = track
        }
    }

    static func dismantleUIView(_ view: VideoView, coordinator: ()) {
        view.track = nil
    }
}

As with VideoView itself, the wrapper does not keep the track: pass in a track that something else holds.