List of translated parts of the series:

1. Cooking
2. Compiling with Emscripten
3. Convert avi to mp4 (you are here)

---

In this part, we will analyze:

1. Compilation of the FFmpeg library with optimized arguments.
2. Emscripten file system management.
3. Development of ffmpeg.js v0.1.0 and video conversion.

---

Compiling the FFmpeg library with optimized arguments

Although the final goal of this part is to create ffmpeg.js v0.1.0 for converting avi to mp4, in the previous part we created only a “bare” version of FFmpeg, which would be nice to optimize with a few parameters.

1. -Oz : optimize the code and reduce its size (from 30 to 15 MB)
2. -o javascript / ffmpeg-core.js : save js and wasm files to javascript directory. (from where we will call ffmpeg-core.js from the ffmpeg.js wrapper library, which provides a beautiful API)
3. -s MODULARIZE = 1 : create a library instead of a command line utility (you will need to modify the sources, details below)
4. -s EXPORTED_FUNCTIONS = "[_ ffmpeg]" : export the ffmpeg C function to the JavaScript world
5. -s EXTRA_EXPORTED_RUNTIME_METHODS = "[cwrap, FS, getValue, setValue]" : additional functions for working with the file system and pointers, details can be found in the article Interacting with code .
6. -s ALLOW_MEMORY_GROWTH = 1 : remove the limit on the consumed memory
7. -lpthread : removed since we plan to create our own worker. (this is a backlog for the fourth part of publications)

More details about each of the arguments can be found in src / settings.js in the github repository emscripten.

When adding -s MODULARIZE = 1, we will need to modify the source code to meet the requirements of modularity (in fact, get rid of the main () function). You have to change only three lines.

1. fftools / ffmpeg.c : rename main to ffmpeg

- int main(int argc, char **argv) + int ffmpeg(int argc, char **argv)
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

2. fftools / ffmpeg.h : add ffmpeg to the end of the file to export the function

 + int ffmpeg(int argc, char** argv); #endif /* FFTOOLS_FFMPEG_H */
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

3. fftools / cmdutils.c : comment out exit (ret) so that our library does not exit the runtime for us (we will improve this point later).

 void exit_program(int ret){ if (program_exit) program_exit(ret); - exit(ret); + // exit(ret); }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Our new version of the compilation script:

 emcc \ -Llibavcodec -Llibavdevice -Llibavfilter -Llibavformat -Llibavresample -Llibavutil -Llibpostproc -Llibswscale -Llibswresample \ -Qunused-arguments -Oz \ -o javascript/ffmpeg-core.js fftools/ffmpeg_opt.o fftools/ffmpeg_filter.o fftools/ffmpeg_hw.o fftools/cmdutils.o fftools/ffmpeg.o \ -lavdevice -lavfilter -lavformat -lavcodec -lswresample -lswscale -lavutil -lm \ -s MODULARIZE=1 \ -s EXPORTED_FUNCTIONS="[_ffmpeg]" \ -s EXTRA_EXPORTED_RUNTIME_METHODS="[cwrap, FS, getValue, setValue]" \ -s TOTAL_MEMORY=33554432 \ -s ALLOW_MEMORY_GROWTH=1
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

ffmpeg-core.js is ready!

If you have experience with ffmpeg, you already know what a typical command looks like:

 $ ffmpeg -i input.avi output.mp4
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

And since we use the ffmpeg function instead of main, the command call will look like this:

 const args = ['./ffmpeg', '-i', 'input.avi', 'output.mp4']; ffmpeg(args.length, args);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Of course, not everything is so simple, we will need to build a bridge between the worlds of JavaScript and C, so let's start with the emscripten file system.

Emscripten file system management

Emscripten has a virtual file system to support reading / writing from C, which ffmpeg-core.js uses to work with video files.

Read more about this in the File System API .

In order for everything to work, we export the FS API from emscripten, which is due to the parameter above:

 -s EXTRA_EXPORTED_RUNTIME_METHODS="[cwrap, FS, getValue, setValue]"
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

To save the file, you need to prepare an array in the Uint8Array format in the environment of Node.js, which can be done something like this:

 const fs = require('fs'); const data = new Uint8Array(fs.readFileSync('./input.avi'));
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

And save it to the emscripten file system using FS.writeFile ():

 require('./ffmpeg-core.js)() .then(Module => { Module.FS.writeFile('input.avi', data); });
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

And to download the file from emscripten:

 require('./ffmpeg-core.js)() .then(Module => { const data = Module.FS.readFile('output.mp4'); });
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Let's start developing ffmpeg.js to hide these difficulties behind a beautiful API.

Development ffmpeg.js v0.1.0 and video conversion

The development of ffmpeg.js is not trivial, as you constantly need to switch between the worlds of JavaScript and C, but if you are familiar with pointers , it will be much easier to understand what is happening here.

Our task is to develop ffmpeg.js like this:

 const fs = require('fs'); const ffmpeg = require('@ffmpeg/ffmpeg'); (async () => { await ffmpeg.load(); const data = ffmpeg.transcode('./input.avi', 'mp4'); fs.writeFileSync('./output.mp4', data); })();
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

First, download ffmpeg-core.js, which is traditionally done asynchronously so as not to block the main thread.

Here's what it looks like:

 const { setModule } = require('./util/module'); const FFmpegCore = require('./ffmpeg-core'); module.exports = () => ( new Promise((resolve, reject) => { FFmpegCore() .then((Module) => { setModule(Module); resolve(); }); }) );
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

It may seem strange that we wrap one promise in another, this is because FFmpegCore () is not a real promise, but just a function that simulates the promise API.

The next step is using Module to get the ffmpeg function using the cwrap function:

 // int ffmpeg(int argc, char **argv) const ffmpeg = Module.cwrap('ffmpeg', 'number', ['number', 'number']);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The first argument to cwrap is the name of the function (which must be in EXPORTED_FUNCTIONS with the previous underscore), the second is the type of the return value, the third is the type of the arguments of the function (int argc and char ** argv).

It’s clear why argc is a number, but why is argv also a number? argv is a pointer, and the pointer stores the address in memory (of type 0xfffffff), so the pointer type is 32 bit unsigned in WebAssembly. This is why we specify the number as the argv type.

To call ffmpeg (), the first argument will be a regular number in JavaScript, but the second argument should be a pointer to an array of characters (Uint8 in JavaScript).

We divide this task into 2 subtasks:

1. How to create a pointer to an array of characters?
2. How to create a pointer to an array of pointers?

We will solve the first problem by creating the str2ptr utility:

 const { getModule } = require('./module'); module.exports = (s) => { const Module = getModule(); const ptr = Module._malloc((s.length+1)*Uint8Array.BYTES_PER_ELEMENT); for (let i = 0; i < s.length; i++) { Module.setValue(ptr+i, s.charCodeAt(i), 'i8'); } Module.setValue(ptr+s.length, 0, 'i8'); return ptr; };
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Module._malloc () is similar to malloc () in C, it allocates a chunk of memory on the heap. Module.setValue () sets the specific value by pointer.

Remember to add 0 to the end of the character array to avoid unforeseen situations.

Having dealt with the first subtask, create strList2ptr to solve the second:

 const { getModule } = require('./module'); const str2ptr = require('./str2ptr'); module.exports = (strList) => { const Module = getModule(); const listPtr = Module._malloc(strList.length*Uint32Array.BYTES_PER_ELEMENT); strList.forEach((s, idx) => { const strPtr = str2ptr(s); Module.setValue(listPtr + (4*idx), strPtr, 'i32'); }); return listPtr; };
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The main thing to understand here is that the pointer is a Uint32 value inside JavaScript, so listPtr is a pointer to a Uint32 array that stores pointers to a Uint8 array.

Putting it all together we get the following implementation of ffmepg.transcode () :

 const fs = require('fs'); const { getModule } = require('./util/module'); const strList2ptr = require('./util/strList2ptr'); module.exports = (inputPath, outputExt) => { const Module = getModule(); const data = new Uint8Array(fs.readFileSync(inputPath)); const ffmpeg = Module.cwrap('ffmpeg', 'number', ['number', 'number']); const args = ['./ffmpeg', '-i', 'input.avi', `output.${outputExt}`]; Module.FS.writeFile('input.avi', data); ffmpeg(args.length, strList2ptr(args)); return Buffer.from(Module.FS.readFile(`output.${outputExt}`)); };
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Done! Now we have ffmpeg.js v0.1.0 for converting avi to mp4.

You can test the result yourself by installing the library:

 $ npm install @ffmpeg/ffmpeg@0.1.0
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

And converting the file like this:

 const fs = require('fs'); const ffmpeg = require('@ffmpeg/ffmpeg'); (async () => { await ffmpeg.load(); const data = ffmpeg.transcode('./input.avi', 'mp4'); fs.writeFileSync('./output.mp4', data); })();
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Just keep in mind that so far the library only works for Node.js, but in the next part we will add support for web-worker (and child_process in Node.js).

Source Codes:

• ffmpeg-core.js
• ffmpeg.js