SharpHLS

Note

This app is a work in progress and while functional, will require changing some hard coded values to work on your machine/in your environment. YMMV.

Description

SharpHLS is an HLS toolkit leveraging SharpJaad (described below). At this time, it is a fairly slim implementation focused only on streaming AAC LC audio from an MP4 container. This encompasses two components (csproj files in this repo):

1. SharpHLS.CLI - this is a CLI app that generates an hlsdat.json file for a given MP4 file. This file contains a breakdown of the HLS segments the API will serve, and which audio frames in the MP4 file will make up that segment. We take advantage of dotnet "self-contained" builds so users can easily run this CLI in an environment without the dotnet runtime installed. Ex: dotnet publish -r linux-x64 --self-contained true

2. SharpHLS.API - this is a standard ASP.NET web app that returns both the HLS metadata and audio segments for a given track, and leverages the hlsdat.json (we currently expect this to live beside the MP4 file) to correctly serve the file.

On Processing

While HLS is a simple protocol, unfortunately there is a bit of legwork involved to go from an MP4 file to valid HLS segments. While the AAC audio frames don't need manipulated, when they are included in an MP4 container, all the necessary metadata about them is housed in the container's header. This is not condusive to streaming however, as a player might start part way through a track and it should not have to load the header every time or hold it in memory.

This is solved by HLS (and some other streaming protocols) by prefixing every audio frame with a 7-byte ADTS header. What this means is that when we build our HLS segments, we not only need to read hundreds of AAC frames from the source file to build up ~5s worth of audio, we also need to add that prefix header to each of them so that they can be played outside of the original MP4 container.

We want our API to be as fast and simple as possible, so our CLI application extracts all the metadata necessary for this processing to what we call an hlsdat.json file. When a customer requests an HLS manifest and subsequently the audio segments themselves, the API already knows both the start and end of the 5second HLS segment (so it can perform a single range read against object storage) and also the start and end of each AAC frame. This allows it to insert the appropriate ADTS headers and thus, serve playable audio for a library like hls.js.

SharpJaad

JAAD is an AAC decoder and MP4 demultiplexer library written completely in Java. It uses no native libraries, is platform-independent and portable. It can read MP4 container from almost every input-stream (files, network sockets etc.) and decode AAC-LC (Low Complexity) and HE-AAC (High Efficiency/AAC+).

SharpJaad is a netstandard2.0 port of the original Java code base with a few small fixes. It has no third-party dependencies and it's portable as it does not use any platform-specific APIs.

SharpJaad.AAC

SharpJaad.AAC is just the AAC decoder without any MP4 code. It's well suited for applications in RTP where the MP4 container is not needed.

Example

To decode stereo AAC Low Complexity (AAC-LC) with 44.1kHz, start with creating the configuration:

var decoderConfig = new DecoderConfig();
decoderConfig.SetProfile(Profile.AAC_LC);
decoderConfig.SetSampleFrequency(SampleFrequency.SAMPLE_FREQUENCY_44100);
decoderConfig.SetChannelConfiguration((ChannelConfiguration)2);

Create the decoder:

var aacDecoder = new Decoder(decoderConfig);

Create an output buffer for the result:

SampleBuffer buffer = new SampleBuffer();
buffer.SetBigEndian(false);

Decode the AAC frame:

byte[] aacFrame = { ... } // AAC frame to decode
aacDecoder.DecodeFrame(aacFrame, buffer);

Convert it to PCM:

short[] pcm = new short[buffer.Data.Length / sizeof(short)];
Buffer.BlockCopy(buffer.Data, 0, pcm, 0, buffer.Data.Length);

SharpJaadMain

A sample application that can extract AAC audio from a (MP4) file and transcode it into a WAV file.

Credits

SharpHLS leverages SharpJaad to parse MP4 containers and AAC audio streams. A fork of https://github.com/jimm98y/SharpJaad is included here.

SharpJaad is a port of the https://sourceforge.net/projects/jaadec/ into C# by Lukas Volf (jimm98y).

License

SharpJaad uses the MIT license (see the LICENSE file).