Smart Home CameraStream Trait Schema

action.devices.traits.CameraStream - This trait covers how to control the camerastream of a device.

This trait belongs to devices which have the capability to stream video feeds to smart displays, Chromecast-enabled devices, or smartphones. By and large, these are security cameras or baby cameras. But this trait also applies to more complex devices which have a camera on them (for example, video-conferencing devices or a vacuum robot with a camera on it).

Device ATTRIBUTES

Devices with this trait may report the following attributes as part of the SYNC operation. To learn more about handling SYNC intents, see Intent fulfillment.

Examples

Camera with multiple protocols support, not requiring an auth token.

{
  "cameraStreamSupportedProtocols": [
    "webrtc",
    "hls",
    "dash",
    "smooth_stream",
    "progressive_mp4"
  ],
  "cameraStreamNeedAuthToken": false
}

Camera supporting a single protocol, requiring an auth token.

{
  "cameraStreamSupportedProtocols": [
    "hls"
  ],
  "cameraStreamNeedAuthToken": true
}

Supported streaming protocols

This trait supports the following streaming protocols:

• webrtc: WebRTC
• hls: HTTP Live Streaming
• dash: Dynamic Adaptive Streaming over HTTP
• smooth_stream: Smooth Streaming
• progressive_mp4: Progressive MP4 (mostly used for clips)

When streaming hls,dash,smooth_stream, and progressive_mp4 protocols to Cast devices (Chromecasts, smart displays, and Chromecast-enabled smart TVs), a Cast Web Receiver is launched to process the stream and render it on the device. It is recommended that the developer create a Custom Web Receiver to enable access to debugging tools, customize player behavior, customize UI branding, and include analytics. To enable the usage of the custom receiver and opt out of using the default receiver, set the receiver app id that is generated when you register your Cast application in the EXECUTE response's cameraStreamReceiverAppId field.

To learn more about Custom Web Receiver applications, visit the Developer's Site guide.

Device STATES

None.

Device COMMANDS

Devices with this trait may respond to the following commands as part of the EXECUTE operation. To learn more about handling EXECUTE intents, see Intent fulfillment.

action.devices.commands.GetCameraStream

Parameters

Results

Non-WebRTC results

WebRTC results

Examples

Show the front door camera (default Cast receiver).

{
  "cameraStreamAccessUrl": "https://fluffysheep.com/baaaaa.mp4",
  "cameraStreamProtocol": "progressive_mp4"
}

Show front door camera (custom Cast receiver).

{
  "cameraStreamAccessUrl": "https://fluffysheep.com/baaaaa.mp4",
  "cameraStreamReceiverAppId": "1g2f89213hg",
  "cameraStreamAuthToken": "12657342190192783",
  "cameraStreamProtocol": "progressive_mp4"
}

Show front door camera (WebRTC stream).

{
  "cameraStreamIceServers": "[{\"urls\": \"stun:stun.l.partner.com:19302\"},{\"urls\":\"turn:192.158.29.39:3478?transport=udp\",\"credential\": \"JZEOEt2V3Qb0y27GRntt2u2PAYA=\",\"username\": \"28224511:1379330808\"},{\"urls\": \"turn:192.158.29.39:3478?transport=tcp\",\"credential\": \"JZEOEt2V3Qb0y27GRntt2u2PAYA=\",\"username\": \"28224511:1379330808\"}]",
  "cameraStreamSignalingUrl": "https://example.com/signaling/answer",
  "cameraStreamOffer": "o=- 4611731400430051336 2 IN IP4 127.0.0.1...",
  "cameraStreamProtocol": "webrtc"
}

Device ERRORS

WebRTC protocol specifications

The benefits of using WebRTC are low latency and 1-way talk. WebRTC uses a POST method with a POST body and response in JSON format.

WebRTC is currently supported on the Google Nest Smart Display and Chromecast with Google TV devices.

This section describes the requirements when using the WebRTC stream protocol.

WebRTC Requirements and Recommendations

• Google currently supports 1-way (half duplex) communication.
• You must support bundling and rtcp-mux.
• You must use (D)TLS 1.2 or later.
• Trickle ICE is not supported. All ICE candidates must be gathered first before sending the SDP.
• It is strongly recommended that you include UDP/IPv4, TCP/IPv4, UDP/IPv6 and TCP/IPv6 ICE candidates to increase the probability of a successful connection.

Supported video resolutions:

• Minimum: 480p
• Maximum: 1080p

Supported video codecs:

• VP8
• H.264

Supported audio codecs:

• Opus (preferred codec)
• G.711/PCMU
• G.722

Cross-Origin Resource Sharing

Cross-Origin Resource Sharing (CORS) is a mechanism that uses additional HTTP Headers to tell browsers to allow a web application running at one origin to access selected resources from a different origin. The server hosting cameraStreamSignalingUrl should respond with the following header:

Access-Control-Allow-Origin: https://www.gstatic.com

Sample signaling request and response

The following example shows a request that Google sends to your signaling service and the corresponding response to Google.

Header:

Authentication: Bearer <cameraStreamAuthToken>
Content-Type: application/json

POST body:

// When camera offer SDP is provided in the execution response, Google provides an answer SDP.
{
  "action": "answer",
  "deviceId": "123",
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// When camera offer SDP is not provided in execution response, Google generates and provides an offer SDP.
{
  "action": "offer",
  "deviceId": "123",
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// Close the current stream session.
{
  "action": "end"
  "deviceId": "123"
}

// Response to accept the answer SDP in the request.
Response Code : 200
{}

// Response to provide the answer SDP from the service provider.
Response Code : 200
{
  // When the camera offer SDP is not provided in the execution response,
  // Google provides the answer SDP via the signaling response.
  "action": "answer"
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// Response to close current session.
Response Code : 200
{}