Add multimedia annotations to PDFs on Android

Along with rich media and video annotations, Nutrient supports a custom gallery format that can display locally stored videos, websites, image galleries, and YouTube videos by using link annotations with URLs using the pspdfkit:// scheme. With this feature, you can securely enrich standard PDF documents with interactive content.

Setup

Depending on the type of content you want to present, Nutrient requires various third-party dependencies, as well as preparations to your project. To allow Nutrient to show image galleries, videos, or websites, you need to declare the PdfMediaDialog activity inside your AndroidManifest.xml:

<activity
    android:name="com.pspdfkit.ui.PdfMediaDialog"
    android:configChanges="orientation|keyboardHidden|screenSize"
    android:theme="@style/Theme.AppCompat.DialogWhenLarge" />

PdfMediaDialog is an activity that is launched as a modal dialog to overlay the viewed PDF. It can host an interactive image gallery, a video player, or a WebView. Unless you only want to show YouTube videos, you must add this activity to your AndroidManifest.xml.

Setting up image galleries

If you want to display image galleries, you have to add the Picasso image loader library as a dependency in your build.gradle file:

dependencies {
    implementation 'com.squareup.picasso:picasso:2.5.2'
}

Setting up YouTube support

By default, Nutrient will open YouTube videos in a WebView. For a more immersive experience when presenting YouTube videos, your app can include the YouTube Android Player API library. Here are the necessary steps for this:

• Register your application in the Google Developer Console to obtain your YouTube API key.

• Download the latest version of YouTubeAndroidPlayerApi.jar and copy it into the libs/ folder of your Android project.

• Add PdfYouTubeActivity and your previously obtained YouTube API key to the manifest, as shown below.

<activity
    android:name="com.pspdfkit.ui.PdfYouTubeActivity"
    android:theme="@style/Theme.AppCompat.DialogWhenLarge"
    android:configChanges="orientation|keyboardHidden|screenSize" />

<meta-data android:name="youtube_api_key" android:value="YOUR_YOUTUBE_API_KEY_GOES_HERE" />

Once the library is included, Nutrient will automatically use it instead of the WebView when playing a YouTube video.

Showing multimedia content

To show multimedia content, your PDF document must contain links with URIs that point to the content using the pspdfkit:// scheme. All URIs follow the same pattern:

pspdfkit://[options]targetPath

All multimedia URIs use the pspdfkit:// scheme. After the scheme, there’s an optional list of options enclosed in square brackets. These options are specific to the linked multimedia content and can be used to configure how it will be displayed. The path should point to the content, either local or remote.

Links can be added to the document in two different ways:

• As a LinkAnnotation carrying a UriAction.

• As an OutlineElement whose action is also a UriAction.

In addition, you can directly execute any given UriAction by passing it to PdfFragment#executeAction.

Image galleries

Nutrient supports showing image galleries that are based on predefined JSON files. For every gallery that should be displayed inside a PDF, there has to be one file with the .gallery file extension inside the app’s assets.

While galleries can be added programmatically, the simplest approach is via a link annotation. A link annotation already defines a bounding box for the position, which the gallery will use. Gallery annotations can be both created and saved in a file or in code. Here’s an example of how to create this in code:

// Create a link on the first page.
val pageIndex = 0
val galleryAnnotation = LinkAnnotation(pageIndex)

// Define the location on the page.
galleryAnnotation.boundingBox = RectF(0f, 0f, 10f, 10f)

// Set the URL pointing to the gallery file.
val url = "pspdfkit://localhost/galleries/sample.gallery"
galleryAnnotation.action = UriAction(url)

// Add the new annotation to the document.
document.addAnnotationToPage(galleryAnnotation)

// Create a link on the first page.
final int pageIndex = 0;
final LinkAnnotation galleryAnnotation = new LinkAnnotation(pageIndex);

// Define the location on the page.
galleryAnnotation.setBoundingBox(new RectF(0f, 0f, 10f, 10f));

// Set the URL pointing to the gallery file.
final String url = "pspdfkit://localhost/galleries/sample.gallery";
galleryAnnotation.setAction(new UriAction(url));

// Add the new annotation to the document.
document.addAnnotationToPage(galleryAnnotation);

The target is a .gallery JSON file that defines the contents and title of each image. Its definition is very simple and looks like this:

[
  {
    "contentURL": "http://www.link-to-image1.com/image1.jpg",
    "caption": "This is a small image."
  },
  {
    "contentURL": "file://android_asset/local_file.jpg",
    "caption": "This is a local image. Captions are optional."
  }
]

contentURL can point to a local or remote source. Image sources can be JPG, PNG, BMP, GIF, or WebP, as well as HEIF on all devices running Android 8 or newer. The image caption is optional.

Currently, .gallery files must be stored inside the app’s assets/ directory, and they must use the .gallery file extension.

Video files

Nutrient allows playback of videos that are stored within your app’s assets/ folder. Videos have to be encoded using one of the supported video formats.

Here’s how you can add a LinkAnnotation to your PDF document for playing back a video file stored inside the app’s assets at assets/videos/small.mp4:

// Create a link on the first page.
val pageIndex = 0
val videoAnnotation = LinkAnnotation(pageIndex)

// Define the location on the page.
videoAnnotation.boundingBox = RectF(0f, 0f, 10f, 10f)

// Set the URL pointing to the video file.
val url = "pspdfkit://localhost/videos/small.mp4"
videoAnnotation.action = UriAction(url)

// Add the new annotation to the document.
document.addAnnotationToPage(videoAnnotation)

// Create a link on the first page.
final int pageIndex = 0;
final LinkAnnotation videoAnnotation = new LinkAnnotation(pageIndex);

// Define the location on the page.
videoAnnotation.setBoundingBox(new RectF(0f, 0f, 10f, 10f));

// Set the URL pointing to the video file.
final String url = "pspdfkit://[autoplay:false,offset:10]localhost/videos/small.mp4";
videoAnnotation.setAction(new UriAction(url));

// Add the new annotation to the document.
document.addAnnotationToPage(videoAnnotation);

All video URLs must begin with pspdfkit://localhost/ (which maps to your app’s assets/ directory), followed by the path of the video relative to your assets root. In the example, video playback is configured to not start automatically after showing the video (i.e. to wait for the user to press play) and to start video playback at second 10.

Available video playback options

Here is a list of all supported options that can be used inside local video playback URLs:

YouTube

To play YouTube videos, provide a link to the video you want to play, but using the pspdfkit:// URL scheme. Available options are the same as for the local videos, but they will be applied only if the video is opened using Nutrient’s improved YouTube support.

Here’s an example URL that will open a YouTube video within Nutrient:

pspdfkit://www.youtube.com/watch?v=JskztPPSJwY

Web content

To display web content in a WebView, simply provide the link using the pspdfkit:// scheme:

pspdfkit://pspdfkit.com