Why attend?
-
No marketing, ever.
Speakers are selected based on their submission, not how much money their company paid; we will never, ever sell a speaking slot. Attendee information isn't for sale either, and that includes any sponsors.
-
Affordable.
We want anyone in the industry to be able to come, which means keeping tickets reasonably priced (thanks largely to our generous sponsors). We also offer free and discounted tickets to students and open source contributors, so please reach out if you're interested.
-
For everyone in the community.
Our community is dedicated to providing an inclusive, enjoyable experience for everyone in the video industry. In this pursuit, and in keeping with our love for reasonable standards, we adopted the Ada Initiative's code of conduct.
Venue and Location
Terra Gallery & Event Venue
511 Harrison St, San Francisco, CA 94105
Terra Gallery & Event Venue is a premiere San Francisco event venue. It is easily accessible via BART and Muni, in a pleasant and clean area of downtown.
But don't just take our word for it. According to one intrepid reviewer: "Terra G is very very clean, airy, well kept and maintained. The kitchen, restrooms and outside space are to my likening as well. Also love the neighborhood, and location is clean and and quiet." …so, it's got to be good.
-
Mike English
Media over QUIC contribution with Rust
In this talk, I will share my practical experience of implementing the Media over QUIC protocol(s) using Rust. With a focus on technical aspects, I aim to provide insights into the challenges, strategies, and benefits of working with Rust in this domain. Additionally, I will discuss the state of the art for Media over QUIC, emphasizing its advantages for broadcast-scale ultra-low latency streaming. During the development process, I anticipate encountering various challenges that are worth reporting on. By sharing firsthand experiences, I aim to provide valuable insights into the pros and cons of using Rust as the language for Media over QUIC implementations. Furthermore, I will address the complexities of implementing a protocol that is still in active development and provide strategies to stay in sync with evolving specifications, even when requirements are a moving target. Key points to be covered in the talk: - Implementation of Media over QUIC protocols using Rust. - Challenges and advantages of using Rust in the context of media streaming. - Strategies for staying in sync with an evolving protocol during active development. - Overview of the state of the art for Media over QUIC and its benefits for ultra-low latency streaming at broadcast scale.
-
Jeremy Doig
Open and Free: How Web philosophy challenged legacy media mindsets - and won
In this talk, Jeremy Doig, the lead of Google/Chrome's media technology direction throughout the launch of the video tag, WebRTC and AOM, will talk about the rationale for taking a different approach, the principles that drove the direction, anecdotes from the trenches - and some exciting challenges that lay ahead for our industry.
-
Ramdas Satyan
AMD
Content Aware Encoding for low latency live streaming encoders using deep learning
Livestreaming has emerged as a captivating medium that is reshaping the way we engage, communicate, and entertain with the world. Platforms like Twitch, YouTube Live, Facebook Live and others have become go-to destinations for audiences seeking real-time experiences, immediate interaction, and the thrill of being part of a live event. Real-time transcoding plays a crucial role in delivering high-quality, compatible, and optimized video content to these audiences using a range of devices and platforms. Most of this transcoding today, occurs using a fixed adaptive bitrate (ABR) ladder that has a predetermined set of bitrate/resolution combinations for each encoded stream. This static or one size fits all approach has the following limitations: content type blindness leading to inefficient use of bandwidth and suboptimal video quality (VQ). To overcome these limitations, Netflix pioneered content aware encoding (or Per-Shot encoding) for VOD use case. The video content is analyzed offline for every shot and efficient encoding decisions such as bitrate, resolution, quantization level is chosen based on convex hull of a given shot (best quality-bitrate points obtained from an ocean of encodes with different parameters) to maximize the VQ while minimizing bandwidth requirements. Live streaming does not have the luxury of “infinite” latency that VOD offers. Real-time transcoding at scale puts additional cap on the processing capacity. This is a challenging problem that has attracted quite a bit of research in recent times. There are several approaches to content aware encoding for low latency encoding. Finding the best possible quality-bitrate trade-off in real-time with available compute while maintaining latency is the name of this game. In this talk, we showcase our work using deep learning (DL) that predicts the “optimal” bitrate for incoming video in real time using data from input and encoder lookahead. We train a fully connected regression network using input statistics (luma histogram) and encoder lookahead statistics (SAD, mv and activity histograms). The ground truth for our purpose of achieving “optimal” bitrate is the bitrate which achieves a minimum VMAF value of 90 (this is the minimum quality bar) for each chosen shot during training. This regression network is very light on compute and can efficiently run without affecting real-time performance or density of the encoder. This network needs a minimum of four frames of lookahead data to produce a high prediction accuracy. We have trained our network to have maximum savings for low complexity content with negligible loss in video quality and bypass very high complexity content. We tested this algorithm using a variety of video clips downloaded from Twitch.tv and here are some of our results: 1. Bitrate savings of more than 30% with < 1 VMAF point degradation for easy content such as talking head and low complexity content. 2. Bitrate savings of 9% on average for medium complexity content with < 1 VMAF point degradation. 3. Negligible savings for high complexity content (as the algorithm knows lowering bitrate would cause VQ degradation). The reasons for using a deep learning-based approach to predict CAE bitrate over traditional approaches are twofold: 1. Nonlinear function produced from DL provides precise bitrate savings without degrading VQ. 2. DL models can be trained/retrained by the content distributor using propriety and specific set to maximize bitrate savings while maintaining high VQ. This approach is applicable for both hardware and software-based encoders which have access to the encoder lookahead statistics mentioned above. These bitrate savings can provide substantial savings on CDN bandwidth and storage costs for content distributors.
-
Josselin Cozanet
Ateme
Towards efficient dynamic Ad insertion in the Edge
Dynamic Ad Insertion (DAI) is a way of seamlessly introducing personalized ads to each client based on its profile. The playlist manipulator responsible for ad insertion is an edge processing operation that analyzes the base playlist and inserts the Ad according to information for an Ad server. This task proved to be very complicated when using HLS (HTTP Live Streaming) in a low latency mode because chunks signalization dramatically increases the number of requests and playlist updates. Moreover, Ads should already be prepared in a chunked mode so that a live player finds the next chunk to display on time. In this context, we present our integration of HLS low latency playlist manipulation associated with a CDN. We managed to handle the time constraints and Ad servers’ communication while remaining conformant to the HLS specification and compliant with most existing players. This solution is a game changer in the Edge video distribution and would additionally have tremendous benefits from the emergent 5G technology thanks to the multi access edge computing (MEC) enabling native cloud technology for the data plane and control plane.
-
Andy Francis
Bitmovin
When IDR Frame Insertion Backfires
Inserting IDR Frames into live streams helps enable clean, seamless transitions when adaptive bitrate streams switch renditions and reduces artifacts that can be introduced by camera switches in a multi-cam production. It can also help provide clean break points for ad insertion and cover slates and in general would be considered a best practice in most of the cases mentioned above. But sometimes, things take an unexpected turn and a best practice turns out to be the root cause of an issue. In this talk, I'll cover 2 very different live stream workflows and incidents where IDR frame insertion caused problems and what could have been done to avoid issues.
-
Guillem Cabrera
Brightcove
Implementing HLS/DASH Content Steering at Scale
Content Steering is a recent addition to both HLS and MPEG DASH standards, enabling dynamic routing of streaming content between different CDNs. Already supported by DASH.js, HLS.js, and several other players, it dramatically simplifies the design of multi-CDN delivery systems. No custom client plugins, DNS redirects, or CMS integrations are needed. However, the deployment HLS/DASH Content Steering mechanism will require additional server-side elements: content steering servers. Such servers must interact directly and frequently with streaming clients - at a TTL interval of about 300 seconds or less. The faster reaction is required, the higher the intensity of client-server exchanges. It implies a high volume of additional requests and raises questions about reliability, scalability, and operating costs. This talk discusses the design of a mass-scale Multi-CDN traffic optimization system specifically addressing this challenge. The proposed solution reduces the logic on the content steering servers to a stateless operation, in which all state variables become parameters of the client-server exchange. Such design enables steering servers to be deployed very inexpensively at the edge (by utilizing edge functions offered by many CDNs or edge platforms). All “heavy” processing, including analysis of playback statistics across all sessions and optimizations considering business logic, is done by other components in the system. Their operation is separated from high-volume real-time decisions made by steering servers and deployed using a conventional cloud platform. This architecture enables mass-scale deployment of Content Steering functions with minimal additional operating costs. In the end, we will present demo and experimental results studying the efficiency of this system. This technology was submitted as an open-source project to the SVTA alliance.
-
Thasso Griebel
castLabs
Navigating HTML5 Encrypted Media Extensions
This presentation embarks on a thorough exploration of HTML5 Encrypted Media Extensions (EME), revealing its practical use cases, intrinsic limitations of core APIs, and potential pitfalls. It aims to unpack EME's operational complexities, spotlighting occasional shortcomings such as cross-platform compatibility issues, constraints on license request controls, and challenges in tracking key statuses through standard APIs. Integral error handling processes associated with the API are also critically scrutinized. Moreover, the talk puts forward innovative solutions designed to both mitigate present obstacles and enhance resilience against possible future challenges. A notable aspect of this presentation is its emphasis on guiding developers through the often convoluted landscape of content encryption, pinpointing key considerations and cautionary advisories. The overarching objective of this talk is to stimulate an enlightening discourse on DRM technologies, with a vision towards nurturing a more robust, future-ready, and universally compatible video encryption framework.
-
Tomas Bacik
CDN77
CDN Challenges of HTTP-based Low Latency Live Streaming Delivery
The presentation goes through the CDN set-up workflow from the origin to the edge and discusses the tweaks and optimisations that need to be done to deliver HTTP low latency (~5s) live streaming without compromising quality. From transcoding and storage all the way to caching and delivery, there are multiple points for optimization in a low-latency streaming stack. This presentation delves into both edge-oriented features, such as custom-built, nginx-based request coalescing, and intuitive cache pre-warming, which allows us to pre-push stream chunks to edges across the globe with multiple quality profiles. These optimizations can drastically reduce initializion time, as well the number of requests hitting origin encoders in peak-heavy scenarios such as large sporting events, etc. Moving towards the origin and encoding, it will explore the possibilities of methods, such as per-title encoding utilizing optimized bitrate ladders based on content-aware convex hull methods. This allows us to serve better-looking content over noticeably lower bitrates, something particularly useful when adapting LL-streaming to users with slower/mobile connection. Throughout our development we have been evaluating and testing which approaches and methods offer substantial effect and which present what we call “diminishing results”. This presentation will introduce what we believe is the key combination of methods for low-latency http-based formats to stay relevant, even in the world of emerging WebRTC-based streaming.
-
Vanessa Pyne
Daily
lo-fi spy: video, steganography, and you
One time I internet searched the term "steganography", thinking it was the study of where dinosaurs lived, but discovered it is actually the practice of hiding secret information inside a non-secret message. There are three qualities of a good steganography approach: imperceptibility, robustness, and embedding capacity. I thought to myself, that sounds like dope spy stuff and I definitely want to do that. It turns out video can be a fantastic non-secret message format in which to conceal covert messages. There are a few common approaches to make data undetectable, including bit manipulation ie the Least Significant Bit algorithm, as well as exploitation of alpha channels, pixel padding, and metadata. Oh and there is this cool cryptographic concept of secret sharing, which can be used in video steganography to ensure the message is retrievable, even if some of the secret bits are missing. And embedding capacity is almost a given, as video files are often large and compressed. This talk will go more in depth into how these methods work but don't worry, the scary LSB and secret sharing maths will be softened with jokes and gifs. Finally, the stretch goal for this spy talk is to encode secret data into a real-time media stream using the WebCodecs API. Early experiments have been less than promising, but I am not yet deterred. Even if real-time video steganography is not achieved, after this talk we can say we know a lot more about this fascinating concept and it is probably more useful than knowledge about dinosaurs*. *dinosaur gifs are likely, but not promised.
-
Andrey Pozdnyakov
Elecard
Unveiling the Illusion: Debunking the Myth of Video Delivery Monitoring on End-User Devices
Operators worldwide are diligently monitoring the delivery quality of their streaming services, often seeking ways to compile monitoring probes within customers' set-top boxes for comprehensive monitoring. However, this talk will shed light on the fact that monitoring customers' set-top boxes is not essential. Instead, the focus of proper monitoring should be on answering the crucial question: Who bears responsibility for malfunctions in the streaming workflow? Installing probes on end user's set-top boxes does not provide an answer to this question. These probes merely testify failure to play the stream on the device, with the same error replicated across all devices in that sector. By delving into the various components of HTTP streaming and exploring common errors in each stage, this talk will emphasize the significance of identifying and addressing issues within the streaming infrastructure itself. Attendees will gain insights into optimizing the streaming workflow, pinpointing responsibility for glitches, and implementing robust monitoring practices at critical stages such as ingestion, transcoding, packaging, CDN, and last mile delivery. The talk is debunking the myth surrounding the need to monitor customers' set-top boxes and equip the attendees with the knowledge and techniques required to achieve seamless streaming experiences through comprehensive infrastructure monitoring based on real-world use cases.
-
Adam Wieckowski
Fraunhofer
Why is your encoder so slow? The curse of next-gen standards
Encoders for next-gen video standard are infamous for being slow. Some of that can be attributed to the immaturity of the software, but it is also the design that makes modern codecs slow for encoding. This talk will explore the inherent properties of next-gen video codecs and show what makes them slow. Beyond that, the talk will show, based on practical experiences from a codec developer, how those issues can be mitigated. The talk will combine abstract high level analytics with practical experiences from the development of an optimized encoder for the recent Versatile Video Coding (VVC) standard. While based on a specific codec and implementation, the concepts generalize to a variety of modern standards, including AV1 and HEVC. Based on the specific example of motion field merge modes in VVC, it will be shown how they are both a blessing for compression efficiency and a curse for encoder search. The underlying reason being, the merge modes represent the toughest kind of search spaces - non-differentable enumerations. After answering the question posed in the title, it will be discussed how to retain most of the coding efficiency without waiting an eternity. Based on specific examples from VVC, the best kind of tools will be explored: those with implicit activation conditions and without signaling.
-
Eric Tang
Livepeer
Unmasking Digital Deception: The Content Authenticity Initiative For Video Engineers
Ever wondered how we can combat the rampant spread of misinformation in video platforms? Enter the Content Authenticity Initiative (CAI), a badass industry-wide effort to establish authenticity and provenance. In this session, we'll geek out over the CAI's technical standard (C2PA) that connects the dots from the capture device all the way to the video editing software, streaming infrastructure, and video player. We'll unravel the mysteries of content provenance, ensuring that what you see is truly what you get. With generative AI on the rise, it's time to arm ourselves with the tools to authenticate digital content. We'll talk about the latest tools available today and how you can integrate it seamlessly into your video tech stack. Get ready to level up your authenticity game!
-
Jordi Cenzano
Meta
Cacheable live under 200ms glass 2 glass. Is it VC or livestreaming? It is MOQ
Media over QUIC (MOQ) is an IETF proposal for a media transport protocol that involves several big players of online video space (Meta, Youtube, Cisco, Akamai, etc). Ideally MOQ can accommodate most the online media use cases we see today: - ULL live streaming (main focus now) - Live streaming - Live rewind - VOD - And Video conferencing!!! To start proving some of the novel ideas of the MOQ working group we developed and open sourced an end to end proof of concept leveraging latest / brightest media technologies such as webCodecs and webTransport. The presentation would: - Introduce MOQ ideas - Start describing end 2 end components - Encoder implementation - Webocodecs, Webtransport, WebWorkers - Relay (CDN) implementation - Based on QUIC-GO - Player implementation - Webocodecs, Webtransport, WebWorkers, audioContext, AudioWorklet, SharedArrayBuffer - Live demo showing latencies <200ms for RTT of ~40ms
-
Justin Sanford
Mux
Journeys through the content moderation landscape of video
Everyone that's worked in video for any amount of time has either encountered, or knowingly ignored, the spectre that is content moderation. If you allow users to upload video to the internet, you're bound to get content that you don't want to host, whether for brand reputation, financial stability, or, in some cases, issues with the law(s). Content rights infringement (whether IP-related like shows on Netflix or sports rights), pornography, violence, and illegal content are among the multitude of things that providers need to be aware of. This talk will start with a high level overview of why content moderation is important. It will then cover some basics of how companies can moderate content. In the end, it will focus on something that we've worked on ourselves, combining live captioning, AI (yes, ChatGPT), and live streaming in order to help detect live sports content that might be problematic. No guarantee is made for how well this works, but it's a fun journey regardless.
-
Matt Szatmary
Mux
JavaScript can be the missing spork in ffmpeg's swiss army knife
ffmpeg has been called the swiss army knife of media processing. And it is a fair comparison. Knife, core screw, tooth pick, even spoon and fork are all there. But if you really need the kitchen sink, you may not find it. Controlling a filter graph is limited to the filters available and the expressiveness of the CLI. But what if ffmpeg allowed you to write custom filters and safely inject them at runtime, and what if the filters could be written in a language you already know?
-
Victoria Nemiah
Mux
Why Your Lawyer is Scared of AI
If you've ever watched a "hacking" scene on TV and wanted to scream "THAT'S NOT HOW ANY OF THIS WORKS!" at the screen, you'll understand where I'm at when it comes to discussion of the laws around AI. From journalistic coverage to HackerNews threads, people have been boldly tossing in incorrect assertions about copyright, privacy, and other complex areas of law. While AI-specific regulation still seems like it's a ways out, there's a lot we can learn from looking at the areas of law that likely do touch AI, especially those that are relevant to video. I'll step us through how copyright law can protect the content ingested into models, how privacy laws are triggered by the use of people's names and faces, how the "right of publicity" could be wielded by celebrities and members of the public alike, and how rules around deceptive trade practices could come into play.
-
Shravya Kunamalla
Netflix
Using application informed pacing to smooth video traffic and be a friendly internet neighbor
On-demand streaming video traffic is managed by an adaptive bitrate (ABR) algorithm whose job is to optimize quality of experience (QoE) for a single video session. ABR algorithms leave the question of sharing network resources up to transport-layer algorithms. TCP is the most widely used transport protocol for streaming video and it is known to send packets in bursts. Traditionally TCP algorithms have focussed on controlling the number of unacknowledged bytes-in-flight at any given time. The process for releasing new bytes typically leads to very bursty traffic on-the-wire. Such burstiness can overload a path’s bottlenecks, leading to excess packet loss, building up queues in network routers, causing excessive delay. This behavior is problematic not only to the video traffic itself, but also to all traffic sharing the same bottleneck link. Additionally, the burstiness causes packets to arrive at clients in bursts, leading to CPU spikes and causing interruptions on CPU-constraint clients. Pacing can help smooth out the burstiness by spreading packet emissions over a period of time while achieving a target sending rate. This gives precise control over the timing of packet emission on-the-wire during per-chunk retrieval on individual TCP connections. Pacing has been around for a while with the first reference around 1991 and the wider TCP community has observed performance benefits from such precise control. In this presentation, we will introduce application-informed pacing, which allows ABR algorithms to set a target pace rate. This rate is decided by the application based on client needs, relayed to the server and it acts as an upper limit on packet-by-packet send rates. We will also propose a joint ABR and rate-control scheme, called Sammy, which selects both video quality and pacing rates. With these two strategies, we will demonstrate that we can substantially smooth video traffic to improve its interactions with the rest of the internet, while maintaining the same QoE for streaming video. We implement our scheme and evaluate it at Netflix. We will show that with no compromise to the streaming QoE, we can improve both retransmissions and RTTs in comparison to existing, extensively tested and tuned production ABR algorithms. Our approach smoothes video, making it a more friendly neighbor to other internet applications.
-
Andrey Norkin
Netflix
Notes on AV1 productization and royalty-free video codecs outlook.
AV1 video coding standard was developed by the Alliance for Open Media (AOMedia) by 2018. The first part of the talk will focus on AV1 productization at Netflix. Rolling out a new codec in production has its challenges, and the benefits should be weighted against the costs. The talk will give an overview of AV1 codec rollout at Netflix, including visual quality aspects, and demonstrate the benefits of using this codec for delivery of Netflix content in VoD. The second part of the talk will offer a glimpse of the research conducted by AOMedia on video compression beyond AV1. It will present the current status of the research and show the performance of its AOMedia video model (AVM) software compared to the state of the art.
-
Kieran Kunhya
Open Broadcast Systems
Baby Demuxed's First Assembly Language Function
Hand written assembly language functions are the bedrock of any real-world codec implementation. It's the reason any online video encodes at a reasonable speed or decodes without looking like a stuttery mess. Yet knowledge of writing assembly language functions is slowly disappearing. This presentation will explain how assembly language functions are important in video codecs and try to teach the audience how to write their first x86 assembly language function. This might sound hard, but high-schoolers have literally written the functions you use every single day.
-
Amy Prosser
Optus Sport
“The Beautiful Game”… It’s not a Man’s World after all
The football narrative is changing. Attendance, Broadcast and streaming records are being smashed and it’s not for the men’s game anymore. This trend isn’t exclusive to football either - Women’s sport is exploding on a global scale and, in a big part, it’s thanks to the work we do. This short talk reflects on some of the ways in which streaming has helped make positive strides for underrepresented sports and will hopefully leave us feeling a bit inspired about the ongoing opportunities our world can create.
-
Kevin Yu
Pluto TV
-
Prabhu Hosur
Pluto TV
-
Spencer Shanson
Pluto TV
Server-initiated client-side just-in-time stitching in ABR streaming
In a typical server-side stitching, complete manifests ( i.e., playlists in HLS and MPD in MPEG-DASH) of different media or Ad content are stitched in a back-end stitcher service. In recent just-in-time stitching technology that is being adopted by the industry, only reference links of media or Ad manifests are provided to the client and actual stitching of the manifests happens on the client side. The reference links for the manifests may be located at different locations. This presentation shows the advantages and disadvantages of different methods of using Interstitials in HLS and X-link in MPEG-DASH for just-in-time stitching on client side. The HLS and MPEG-DASH just-in-time stitching requires client-side implementation support. This presentation walks through the additional software modules that are integrated to support just-in-time stitching in Safari browser, Apple device simulator, DASH.js player etc.
-
Nicolas Gonzalez
Qualabs
The Bug Hunter's Guide: Unraveling Video Players CMCD implementation in a month
[(This is a fine tunning of a talk named: How we were able to create a tool for the video dev community in a month that found bugs in major video players.)] At Qualabs we hosted Monte<Video> Summer camp this february, it was a community event focused on connection ("CONNECTING THROUGH THE POWER OF COLLABORATION"). We wanted other video devs to join us in our offices in Montevideo, to experience our culture, our summer, our offices, and why not, create something together. So we asked the community, what ideas are interesting for us to build? What do we need? We had two very interesting proposals: Will Law from Akamai proposed a CMCD Validator tool that could help developers adopt the standard easily. Marco Vidonis from Nimble ape proposed to create an open source hybrid meetup platform. These projects would be publicly available to everyone who would want to join, develop and learn in the process. Everything about the management of the project would also be public and we would have in person meetings when possible. The focus of this talk is on the CMCD Validator project since in that month of work, we were able to create a tool that would enable any video developer to use as part of their development flow to incorporate the CMCD standard on the players. Also it would allow us to test any existing implementation of CMCD in the players. We found interesting things: Reported CMCD implementation bugs on: Dash.js Bitmovin Shaka player We proposed changes on the CMCD standard to clear ambiguity. After the summer camp ended, we kept gathering ideas and optimizations to add to the tool. We kept working on it, now it is able to start analyzing differences between how players handle the different parameters, it has its own analyzer tool integrated with elastic and Kibana and can be used as a "proxy" to be able to capture information from mobile devices. We also used it to build a CMCD course for SVTA, using the tool to generate examples and showcase good practices. But, what is in it for the devs? The creation of this tool was a great way to involve our developers more into the community. By hosting this event, the devs involved had to be aware of what the community liked, how to ask for their feedback, and how to report bugs on the players with the right information, and how to make the life of other devs easier. They also learned a lot about specifics for the CMCD standard and how important it is in the ecosystem improving their skills. For closing this talk, we also want to talk about how we can make this a muscle. We definitely want to be able to create more tools/code in the next iterations of the Monte<video> Summer camp. Can we keep creating these in other spaces? Can we start showing the power of the community in all our community events? What more can we create?
-
Luke McGartland
Sequence
Exploring Video Editing in the Browser
Many tools for manipulating video in the browser are appearing every day: Sequence, Scenery, Veed.io, Runway, etc. How do they work? What are the challenges of browser based editing? How will new APIs like WebCodecs, WebGPU change the landscape? What are the tradeoffs of rendering video in the DOM vs using <canvas/>? This talk will explore the challenges of how to manipulate video in the browser and the future of browser based editing. Speaker has founded a browser based editing tool and has extensive knowledge of the space.
-
Alvaro Laserna Lopez
TestDevLab
Using AI to get relevant KPIs for Audio Quality
Description: Good metrics for audio quality are rare to find, and some of them doesnt give too much information about what's the problem, or wether the quality is really one way or another. Some examples are POLQA or ViSQOL, but those two are algorithms requires the reference, and the degraded audio samples to be complete (full reference). We are proposing two new algorithms based on Deep Learning, and specifically on transformers. The first one is purely focused on speech transcription (SpeechQ) and the second one is about audio classification using audio spectrogram transformers (ASQ-ViT), but using a convolutional neural network as a backbone for feature extraction. The results from this training has lead to a very high correlation with subjective scoring, which proves the effectiveness and simplicity of this algorithm to extract an objective quality measurement from audio outputs.
-
Pieter-Jan Speelmans
THEOplayer
Stories from the trenches: debugging video issues from a client’s perspective
I have accumulated over a decade of experience in the realm of video player development. Interestingly, when things go awry, the knee-jerk reaction tends to be, "It must be the player's fault." However, I can reveal a spoiler: More often than not, the player isn't the culprit. Navigating the intricacies of media can be a daunting task. This complexity is amplified when you have only a limited perspective on the entire architectural stack. Thankfully, video players often serve as the convergence point on the client side, offering a treasure trove of information that, if properly harnessed, can lead you to the root of the problem—if only you knew where to look. Today, many companies lean heavily on analytics dashboards to pinpoint issues, which is undeniably valuable. However, in practice, we frequently encounter scenarios where these tools seem to work against us rather than for us when we're attempting to identify the root causes of problems. Another prevalent issue is the prolonged turnaround time when addressing issues in the field, particularly when dealing with specific customers, devices, or locations. This challenge becomes even more pronounced when trying to replicate issues using a reference app that lacks the genuine threading, UI, and environment. I'm going to make the case that there's no substitute for the effectiveness of interactive debugging in a live production environment.
-
Mattias Buelens
TheoPlayer
The curious player of Benjamin Button: reverse video on the web
The <video> element is the foundation of all video experiences on the web. Yet one key feature is still missing in most browsers: playing a video in reverse. The HTML specification defines how a negative playback rate should be handled, but only a single browser actually implemented this (thanks Safari!). This lack of support is a problem for advanced use cases, such as video editing or subtitle authoring. In this talk, I will explore how to make a video play in reverse inside a web browser. Since a regular <video> element won’t do the trick, I’ll take control of decoding and rendering the individual frames using WebCodecs. I’ll dive deeper into some of the challenges that arise when playing in reverse, such as: * How to keep playback smooth without stuttering. * How to buffer enough GOPs, without stalling the hardware decoder. * How to adapt a streaming video player’s download loop to support buffering in reverse.
-
Jon Kurinsky
Tubi
Client-Side Thumbnailing With WebCodecs
In cases where we can't wait for async generation of thumbnails (say, presenting user-submitted content), there is sometimes a need to extract thumbnails from encoded content in the browser. For years, the default solution has involved stepping through the content in a hidden video element, and copying frame data to a canvas. But now that the WebCodecs proposal is moving toward wider availability, more performant solutions are available. This lightning talk will demonstrate one such technique and its improved performance relative to stepping through video using the HTML video element.
-
Todd Sharp
Twitch
Experimenting with Hardware and AI to Create Dynamic Live Streaming User Experiences
In this session, we'll take a look at various hardware and AI based experiments aimed at creating unique user experiences to enhance live streaming applications. What if we could truly automate multi-cam live streams and make them accessible to both UGC and PGC streamers? In this session, we’ll look at one solution for automated multi-cam streams by utilizing millimeter wave human presence sensors to automatically composite multiple cameras or switch a stream’s source camera when appropriate. What about using computer vision to analyze live stream chat or even frames to determine a broadcaster or viewers sentiment? We'll look at a real life "mood ring" that changes colors based on the sentiment as determined by AI/ML. Have you ever looked into PTZ cameras for your live streams but realized just how expensive they are? We'll see how to build your own PTZ camera with some hardware and a video game controller. These experiments might not be practical or cost effective, but they'll inspire you to think beyond traditional approaches and add new levels of interactivity to your live streaming applications.
-
Ronald S. Bultje
Two Orioles
Low-level wizardry in dav1d
In a world of datacenter virtualization and high-level languages, today we take a peek behind the curtain and watch how the Wizard of Oz does his magic: let’s dive into the world of low level optimizations in dav1d - Videonlan’s software AV1 decoder. First, we will dig into dav1d's AVX512 optimizations for Intel’s most recent CPUs: IceLake and TigerLake. Historically, video encoders and decoders have had issues optimizing for Intel's AVX512 instruction set. The wider vector length should in theory improve performance, but prior to IceLake, the associated clockspeed frequency penalties and lack of interesting new instructions meant that most applications saw little gains. IceLake brought about significant reductions in clockspeed penalties, and new video codecs (like AV1) utilize bigger block sizes: ideal conditions to leverage AVX512’s wider vectors. More importantly, by MacGyvering the new cross-lane shuffle, multiply-accumulate and cryptographic instructions supported in IceLake’s AVX512 subset, we've reached up to 3x speedups in our new AVX512 functions compared to their AVX2 counterparts. Overall, we see a 10% speedup in a fully optimized AVX512 decoder vs. using AVX2 instructions on the same machine. Second, we describe our redesigned threading model. Current (tile- or frame-) threading models need many resources (threads, memory) to saturate a limited number of cores, scale poorly across different-throughput cores (like big.LITTLE) and depend on bitstream features which negatively affect compression. Our new threading model scales regardless of specific features present in the bitstream, requires limited system resources and is ideal for big.LITTLE core combinations that are popular in today’s latest mobile devices. With this design, dav1d can keep all your cores busier than a Barista during a Demuxed break, on ARM as well as x86. 720p real-time AV1 software playback on the large majority of Android devices out in the wild is now a reality.
-
Willy R. Vasquez
University of Texas
Having H26Fun with H26Forge: Vulnerability Hunting, Datamoshing, and More!
Description Modern video encoding standards such as H.264 are a marvel of hidden complexity. But with hidden complexity comes hidden security risk. Decoding video today involves interacting with dedicated hardware accelerators and the proprietary, privileged software components used to drive them. The video decoder ecosystem is obscure, opaque, diverse, highly privileged, largely untested, and highly exposed -- a dangerous combination. We introduce H26Forge, a framework that carefully crafts video files to expose edge cases in H.264 decoders. H26Forge's key insight is operating on the syntax elements rather than on the encoded bitstring to build syntactically correct but semantically spec-non-compliant video files. These files cause H.264 decoders to find themselves in undefined states or unhandled errors. We used H26Forge to uncover numerous vulnerabilities across the video decoder ecosystem, including kernel memory corruption bugs in iOS, memory corruption bugs in Firefox and VLC for Windows, and video accelerator and application processor kernel memory bugs in multiple Android devices. These bugs have been acknowledged by multiple vendors including Apple, Mozilla, and FFmpeg. In this talk, we will describe how H26Forge can be used to find issues that may arise in processing User Generated Content so that video engineers can protect their infrastructure. We will also discuss other uses of H26Forge, such as debugging encoded videos or datamoshing.
-
Jean-Baptiste Kempf
VideoLAN
Kyber: a new approach for real-time video and controls streaming based on Quic
While we have good solutions for low-latency streaming through CDN (Dash, LL-HLS) and for people to people communication (WebRTC), there are not many good open source options for controlling machines, aka video streaming + input streaming multiplexed. Kyber is an open source project comprising low-latency encoder, decoder and protocol stack, based on Quic, to address this. It is flexible, and can be used for Remote Desktop, Cloud Gaming, Drone and Robots control or Application streaming. And a contrario from existing projects, this is cross-platform, for both client and server sides. We've achieved 20/25ms glass-to-glass latency, we've used H.264/HEVC/VP9/AV1 for streaming, and we've been porting the software on Windows/Linux/macOS for the server and Windows/Linux/macOS/iOS/Android. The project is open source and is based on FFmpeg and VLC.
-
Quentin Mazars-Simon
Vimeo
The NeverEnding Story of replacing your transcoding pipeline
Back in 2019, we set upon replacing our aging transcoding pipeline. We’re in 2023 and the old system still hasn’t been deprecated. What is taking us so long? In this talk, we’ll go over why we decided to build a new transcoding platform (goals, why not just improve the existing one, etc.), the challenges we faced (video, infrastructure or otherwise), how it (carefully) got rolled out to our users, and some lessons learned along the way.
Our Speakers
-
Adam Wieckowski
Fraunhofer
-
Alvaro Laserna Lopez
TestDevLab
-
Amy Prosser
Optus Sport
-
Andrey Norkin
Netflix
-
Andrey Pozdnyakov
Elecard
-
Andy Francis
Bitmovin
-
Eric Tang
Livepeer
-
Guillem Cabrera
Brightcove
-
Jean-Baptiste Kempf
VideoLAN
-
Jeremy Doig
-
Jon Kurinsky
Tubi
-
Jordi Cenzano
Meta
-
Josselin Cozanet
Ateme
-
Justin Sanford
Mux
-
Kevin Yu
Pluto TV
-
Kieran Kunhya
Open Broadcast Systems
-
Luke McGartland
Sequence
-
Matt Szatmary
Mux
-
Mattias Buelens
TheoPlayer
-
Mike English
-
Nicolas Gonzalez
Qualabs
-
Pieter-Jan Speelmans
THEOplayer
-
Prabhu Hosur
Pluto TV
-
Quentin Mazars-Simon
Vimeo
-
Ramdas Satyan
AMD
-
Ronald S. Bultje
Two Orioles
-
Shravya Kunamalla
Netflix
-
Spencer Shanson
Pluto TV
-
Thasso Griebel
castLabs
-
Todd Sharp
Twitch
-
Tomas Bacik
CDN77
-
Vanessa Pyne
Daily
-
Victoria Nemiah
Mux
-
Willy R. Vasquez
University of Texas
The Schedule
-
Opening Remarks
Matt McClure
-
Jeremy Doig
-
Amy Prosser
Optus Sport
-
Break
-
Pieter-Jan Speelmans
THEOplayer
-
Adam Wieckowski
Fraunhofer
-
Todd Sharp
Twitch
-
Kieran Kunhya
Open Broadcast Systems
-
Lunch, sponsored by:
-
Mike English
-
Kevin Yu
Pluto TV
Prabhu Hosur
Pluto TV
Spencer Shanson
Pluto TV
-
Andy Francis
Bitmovin
-
Tomas Bacik
CDN77
-
Break
-
Shravya Kunamalla
Netflix
-
Ronald S. Bultje
Two Orioles
-
Victoria Nemiah
Mux
-
Break
-
Josselin Cozanet
Ateme
-
Mattias Buelens
TheoPlayer
-
Thasso Griebel
castLabs
-
Quentin Mazars-Simon
Vimeo
-
Women and Enby Breakfast
RSVP now!
If you like free breakfast and great company, join us for a community-organized networking breakfast -- all who identify as women, non-binary folks, and/or trans folks are welcome. We'll dive into a mix of topics, so bring your Qs & As! -
Andrey Norkin
Netflix
-
Willy R. Vasquez
University of Texas
-
Nicolas Gonzalez
Qualabs
-
Alvaro Laserna Lopez
TestDevLab
-
Break
-
Matt Szatmary
Mux
-
Vanessa Pyne
Daily
-
Ramdas Satyan
AMD
-
Eric Tang
Livepeer
-
Lunch, sponsored by:
-
Lightning Talks
Submit a Lightning Talk
Hard cap of 3 minutes, ordering will be announced the afternoon of the 24th.
-
Break
-
Jean-Baptiste Kempf
VideoLAN
-
Luke McGartland
Sequence
-
Andrey Pozdnyakov
Elecard
-
Guillem Cabrera
Brightcove
-
Break
-
Justin Sanford
Mux
-
Jordi Cenzano
Meta
-
Surprise
-
Closing Remarks
Matt McClure
-
After Party
After 2 long days of talks, we'll head all the way to...the downstairs of the venue! Put your feet up, grab some refreshments, and hang out with your fellow video engineers. Sponsored by:
About us
Demuxed is video engineers talking about video technology. Our first meeting was a single day event back in 2015, born out of the SF Video Technology meetup. The video industry had plenty of trade shows and other opportunities for The Business, but our goal was to create a conference and community for the engineers building the technology powering video, from encoding, to delivery, to playback, and beyond. We've grown a lot since then, but our goal remains the same.
After creating Demuxed, some of the organizers went on to start and work at Mux. Mux continues to sponsor most of the organizational work behind the scenes (thanks for the salary!), but Demuxed is, at its core, a community-led event.
Every year we get a group together that's kind enough to do things like schedule planning, help brainstorm cool swag, and, most importantly, argue heatedly over which talk submissions should make the final cut. These folks are the ones hanging out in Video Dev Slack, and they hail from all over the industry.
