WebRTC Roadmap
#10372
Replies: 1 comment 1 reply
-
|
could you add also the PR for adding HEVC support #10159 |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
done! |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
This is the roadmap/plans I currently have for WebRTC support in OBS. I am posting it here to get feedback from the OBS maintainers and OBS users. If you are interested in contributing we would love to have you! If anyone is interested in live discussion we also have a Realtime-Broadcasting Discord here
We also have a companion project Broadcast Box that these features are coded/tested against.
After this condensed list I have expanded reasoning/explanation of the features.
setLocalDescriptionAdd Mbed TLS support to libdatachannel
Mbed TLS is the DTLS library of choice on Windows. libdatachannel needed to support Mbed TLS to work on OBS Windows
Add WHIP support to OBS
WHIP allows a user to broadcast via WebRTC. These are the benefits I am excited about
Add AV1 WHIP Support
AV1 is better quality at a lower bitrate. It is also has no patent restrictions. It would be great to see users off of H26x and RTMP which both have had litigation associated with them.
Simulcast Support
Run multiple encoders for a single stream. Allows anyone to generate multiple renditions of a video stream and they can be higher quality. This also will make it easier for smaller operators to run servers. You can see Twitch's take on Simulcast with their post on Enhanced Broadcasting
More Certificate Fingerprints to libdatachannel
libdatachannel supported the most common algorithm sha-256. This is used by browsers and most servers. mediasoup uses sha-512 and I only started getting reports after the first release went out. Adding this means more servers work against OBS.
Add version macros to libdatachannel
This allows OBS to conditionally enable features without breaking users stuck on old versions of libdatachannel
Add H264/Opus Depacketizer to libdatachannel
libdatachannel was able to send, but not receive media. Adding this was a prerequisite for WHEP support
Add STUN/TURN Support
We should parse the list of ICE Servers in the WHIP/WHEP response. This contains a list of URLs + credentials. Currently we ignore this entirely. This functionality isn't used by many, but is still part of the spec and users have requested it!
Add WHEP Support to OBS
WHEP allows users to subscribe to WebRTC streams of video.
This change solves these problems for streamers
Podcasting/Conferencing inside of OBS Using WebRTC streamers don't have to leave OBS anymore to build co-streaming. Users can publish their webcam via WHIP, and then pull the via WHEP the users they want to stream with. The WHEP sources can also be generated from non-OBS tools. You could pull a WHEP source that is generated by a guest that is just using the browser.
The quality of these broadcasts will be much higher then screen recording conferencing software. You won't have multiple rounds of encode/decode/composite that add generational loss.
Easier to build multi-computer streaming setups Users can play their game and capture on one computer. Then stream to another computer in their network build the final stream. The low latency of WebRTC means streamers will have less delay between gameplay and commentary. The P2P model of WebRTC also makes this easier.
libdatachannel and RTX
WebRTC is able to achieve such low latency because it runs over UDP. Because we are running over UDP packets may get dropped. WebRTC as a protocol has a feature called RTX where a lost packet can be requested by the receiver, so the lost packet can be fixed.
At this time I am not sure if libdatachannel's RTX support is working. This needs to be investigated. It could be a Broadcast Box issues, it could be our usage of libdatachannel or it could be a libdatachannel issue.
libdatachannel pacing MediaElement
Pacing spreads video sending evenly over time. Currently we are sending video packets as soon as they are available. This puts variable pressure on all the hops between OBS and the receiver. This variable pressure leads to higher packet loss and delay.
We need to add a MediaElement to libdatachannel that adds pacing. Code changes to OBS will be minimal, just enabling this new libdatachannel feature.
Single Source WHIP
Single Source WHIP allows a stream to publish a single source (like a webcam) via WHIP. This would allow users to build co-streaming experiences without having to leave OBS/use conference software. The workflow would look like this
User A and User B can now communicate with each other and they each only have OBS open. Both users then can arrange/design their scene as they wish and go live at any time. This allows users to co-stream, but have a broadcast unique to their channel.
Forward Error Correction
Forward Error Correction allows users to have low latency streams even in adverse networks. Instead of delaying the stream to correct errors in the video WebRTC will send redundant data. When an error occurs that redundant data is used instead of re-requesting lost/broken video.
This is a much better experience for IRL streamers then (N)ACK based protocols because it removes the variable latency.
Optional Bandwidth Estimation
Bandwidth estimation would allow OBS to automatically increase (or decrease) the bitrate of the stream depending on how much network is available. If a user is in a dynamic network situation setting a static bitrate doesn't provide an acceptable experience. If the network temporarily degrades they will see additional delay/packet loss. Instead it would be better to have OBS adjust the bitrate temporarily until the network improves.
This needs investigation if it is better to use the Bandwidth Estimator used by RTMP, or if adding a WebRTC specific one to libdatachannel is better.
Stats output
Provide a visualization (or export) of client side WebRTC stats. This would allow users to see things like
Beta Was this translation helpful? Give feedback.
All reactions