Video Redaction

WebViewer Video uses ffmpeg in the background to allow for redaction of video frames. In order to utilize ffmpeg, a server must be setup to allow for it. We have created a sample server for you to use and deploy. We recommend hosting the server on AWS.

Before you begin, make sure your development environment includes Node.js, npm and ffmpeg.

The WebViewer-Video github sample has a redaction sample ready to go. It will initialize and run the client and server components. We recommend running that sample directly. To do so, first clone repository and then run the following commands:

# command line
npm i
npm run start-samples

The steps below will recreate the sample. Use them to integrate redaction into you application.

WebViewer Video uses ffmpeg in the background to allow for redaction of video frames. In order to utilize ffmpeg, a server must be setup on the user side to allow for it. A sample server can be found here. Please follow installation steps there before continuing. The example code below shows how to integrate the server within your application:

Before you begin, make sure your development environment includes Node.js, npm and ffmpeg.

Below is an example of how to integrate the server with WebViewer Video. In this example, we used update element to overload the click event of the Redact All button. This click event will now make a call to our custom server where the redaction of the video will take place with ffmpeg.

import WebViewer from '@pdftron/webviewer';
import { initializeVideoViewer } from '@pdftron/webviewer-video';
WebViewer({
    path: '/webviewer/lib',
    enableRedaction: true,
  },
  viewer.current,
).then(async instance => {
  // Extends WebViewer to allow loading HTML5 videos (.mp4, ogg, webm).
  const {
    UI,
    loadVideo,
  } = await initializeVideoViewer(
    instance,
    {
      license: '---- Insert commercial license key here after purchase ----',
      enableRedaction: true,
    }
  );
  // Load a video at a specific url.
  // Can be a local or public link
  const videoUrl = 'https://pdftron.s3.amazonaws.com/downloads/pl/video/bunny-short.mp4';
  loadVideo(videoUrl);

  // Overloading the onclick function of redacting video button with custom code
  // Function must be async and must return data when function is finished to remove loading spinner.
  // enableRedaction must be set to true when initalizing WebViewer Video:
  // https://www.pdftron.com/api/video/module-@pdftron_webviewer-video.html#.initializeVideoViewer__anchor
  UI.updateElement('redactApplyButton', {
    onClick: async redactAnnotations => {
      const response = await fetch('http://localhost:3001/video/redact', {
        method: 'POST',
        body: JSON.stringify({
          intervals: redactAnnotations.map(annotation => ({
            start: annotation.getStartTime(),
            end: annotation.getEndTime(),
            shouldRedactAudio: annotation.shouldRedactAudio || annotation.redactionType === 'audioRedaction',
            shouldRedactVideo: annotation.redactionType !== 'audioRedaction',
          })),
          url: videoUrl,
        }),
        headers: {
          'Accept': 'application/json',
          'Content-Type': 'application/json'
        },
      });

      const videoBuffer = await response.arrayBuffer();

      const newVideoBlob = new Blob([videoBuffer], { type: 'video/mp4' });
      loadVideo(URL.createObjectURL(newVideoBlob));
      return videoBuffer;
    }
  });
});

Currently our server returns the video file by sending back a buffer to the client. You will find that code, in this file.

fs.readFile(`./tmp/${uuid}.mp4`, (err, data) => {
  resolve(data);
});

The issue here is that when redactions are applied to this buffer, we cannot send that local file back to the server so it must be persisted on the server through a variable or by uploading to the cloud in order to have further redactions applied to a previously edited video.

Here is an example of replacing the server code above with an upload to s3:

New server code:

return new Promise((resolve) => {
  fs.readFile(`./tmp/${uuid}.mp4`, async (err, data) => {
    await s3.upload({
      Bucket: 'pdftron-media-demo-files',
      Key: `${uuid}.mp4`,
      Body: data
    });

    let s3URL = await s3.getSignedUrl({
      Bucket: 'pdftron-media-demo-files',
      Key: `${uuid}.mp4`,
    });

    resolve(s3URL);
  });
});

New client code:

import WebViewer from '@pdftron/webviewer';
import { initializeVideoViewer } from '@pdftron/webviewer-video';
WebViewer({
    path: '/webviewer/lib',
    enableRedaction: true,
  },
  viewer.current,
).then(async instance => {
  // Extends WebViewer to allow loading HTML5 videos (.mp4, ogg, webm).
  const {
    UI,
    loadVideo,
  } = await initializeVideoViewer(
    instance,
    {
      license: '---- Insert commercial license key here after purchase ----',
      enableRedaction: true,
    }
  );
  // Load a video at a specific url.
  // Can be a local or public link
  let currentVideoUrl = 'https://pdftron.s3.amazonaws.com/downloads/pl/video/bunny-short.mp4';
  loadVideo(currentVideoUrl);

  // Overloading the onclick function of redacting video button with custom code
  // Function must be async and must return data when function is finished to remove loading spinner.
  // enableRedaction must be set to true when initalizing WebViewer Video:
  // https://www.pdftron.com/api/video/module-@pdftron_webviewer-video.html#.initializeVideoViewer__anchor

  UI.updateElement('redactApplyButton', {
    onClick: async redactAnnotations => {
      const response = await fetch('http://localhost:3001/video/redact', {
        method: 'POST',
        body: JSON.stringify({
          intervals: redactAnnotations.map(annotation => ({
            start: annotation.getStartTime(),
            end: annotation.getEndTime(),
            shouldRedactAudio: annotation.shouldRedactAudio || annotation.redactionType === 'audioRedaction',
            shouldRedactVideo: annotation.redactionType !== 'audioRedaction',
          })),
          url: currentVideoUrl,
        }),
        headers: {
          'Accept': 'application/json',
          'Content-Type': 'application/json'
        },
      });

      currentVideoUrl = await response.text();
      loadVideo(currentVideoUrl);
      return currentVideoUrl;
    }
  });
});

Now when using the endpoint /video/redact, your redacted video will be uploaded to s3 and changes will be persisted on the client.