Audio Redaction

WebViewer Audio uses ffmpeg in the background to allow for redaction of video frames. (Check out audio redaction demo to try it now.) In order to utilize ffmpeg, a server must be setup to allow for it. Please follow installation steps on the sample mdeia server before continuing.

We recommend hosting the server on AWS. The example code below shows how to integrate the server within your application:

Before you begin, make sure your development environment includes Node.js, npm and ffmpeg.

Below is an example of how to integrate the server with WebViewer Audio. In this example, we used update element to overload the click event of the Redact buttons. This click event will now make a call to our custom server where the redaction of the video will take place with ffmpeg.

import WebViewer from '@pdftron/webviewer';
import { initializeAudioViewer } from '@pdftron/webviewer-audio';
WebViewer({
    path: '/webviewer/lib',
    enableRedaction: true,
  },
  viewer.current,
).then(async instance => {
  // Extends WebViewer to allow loading media files (.mp3, .mp4, ogg, webm, etc.)
  const {
    loadAudio,
    UI,
  } = await initializeAudioViewer(
    instance,
    {
      license: '---- Insert commercial license key here after purchase ----',
      enableRedaction: true,
    }
  );
  // Load a media element at a specific url.
  // Can be a local or public link
  const audioUrl = 'https://pdftron.s3.amazonaws.com/downloads/pl/video/audio.mp3';
  loadAudio(audioUrl);

  // Overloading the onclick function of redacting audio button with custom code
  // Function must be async and must return the new url when function is finished to remove loading spinner and load new file.
  // enableRedaction must be set to true when initalizing WebViewer Video:
  // https://docs.apryse.com/api/audio/module-@pdftron_webviewer-audio.html#.initializeAudioViewer__anchor
  UI.updateElement('redactApplyButton', {
    onClick: async redactAnnotations => {
      const response = await fetch('http://localhost:3001/audio/redact', {
        method: 'POST',
        body: JSON.stringify({
          intervals: redactAnnotations.map(annotation => ({
            start: annotation.getStartTime(),
            end: annotation.getEndTime(),
          })),
          url: audioUrl,
        }),
        headers: {
          'Accept': 'application/json',
          'Content-Type': 'application/json'
        },
      });

      const audioBuffer = await response.arrayBuffer();
      const newAudioBlob = new Blob([audioBuffer], { type: 'audio/mp3' });
      return URL.createObjectURL(newAudioBlob);
    }
  });
});

Currently our server returns the audio file by sending back a buffer to the client. You will find that code, in this file.

fs.readFile(`./tmp/${uuid}.mp3`, (err, data) => {
  resolve(data);
});

The issue here is that when redactions are applied to this buffer, we cannot send that local file back to the server so it must be persisted on the server through a variable or by uploading to the cloud in order to have further redactions applied to a previously edited audio.

Here is an example of replacing the server code above with an upload to s3:

New server code:

return new Promise((resolve) => {
  fs.readFile(`./tmp/${uuid}.mp3`, async (err, data) => {
    await s3.upload({
      Bucket: 'pdftron-media-demo-files',
      Key: `${uuid}.mp3`,
      Body: data
    });

    let s3URL = await s3.getSignedUrl({
      Bucket: 'pdftron-media-demo-files',
      Key: `${uuid}.mp3`,
    });

    resolve(s3URL);
  });
});

New client code:

import WebViewer from '@pdftron/webviewer';
import { initializeAudioViewer } from '@pdftron/webviewer-audio';
WebViewer({
    path: '/webviewer/lib',
    enableRedaction: true,
  },
  viewer.current,
).then(async instance => {
  // Extends WebViewer to allow loading media files (.mp3, .mp4, ogg, webm, etc.)
  const {
    loadAudio,
    UI,
  } = await initializeAudioViewer(
    instance,
    {
      license: '---- Insert commercial license key here after purchase ----',
      enableRedaction: true,
    }
  );
  // Load a media element at a specific url.
  // Can be a local or public link
  let currentAudioUrl = 'https://pdftron.s3.amazonaws.com/downloads/pl/video/audio.mp3';
  loadAudio(currentAudioUrl);

  // Overloading the onclick function of redacting audio button with custom code
  // Function must be async and must return the new url when function is finished to remove loading spinner and load new file.
  // enableRedaction must be set to true when initalizing WebViewer Video:
  // https://docs.apryse.com/api/audio/module-@pdftron_webviewer-audio.html#.initializeAudioViewer__anchor
  UI.updateElement('redactAudioButton', {
    onClick: async redactAnnotations => {
      const response = await fetch('http://localhost:3001/audio/redact', {
        method: 'POST',
        body: JSON.stringify({
          intervals: redactAnnotations.map(annotation => ({
            start: annotation.start,
            end: annotation.end,
          })),
          url: currentAudioUrl,
        }),
        headers: {
          'Accept': 'application/json',
          'Content-Type': 'application/json'
        },
      });

      const currentAudioUrl = await response.text();
      return URL.createObjectURL(newAudioBlob);
    }
  });
});

Now when using the endpoint /audio/redact, your redacted audio will be uploaded to s3 and changes will be persisted on the client.