Optimizing Annotation Drawing Performance

One of the most important features of PDFs, apart from their immutability, is the ability to add annotations to them. And of all the possible annotation types, ink annotations are one of the most common due to their versatility — they can be used to draw or to add text in the form of a user’s own handwriting.

A PDF can contain thousands of annotations, but naturally, having a large number of annotations has an impact on performance. So, in line with our core value of continuous improvement, our iOS team decided to look into how we could optimize ink annotations.

Investigation and Problems

Recently, when inspecting a PDF with a complex ink annotation, we discovered it was backed by a decent amount of data in the form of points that denote where the ink annotation should be drawn on a page — which is not entirely surprising.

Newer devices have a higher touch sampling rate, which is how often a screen can sense user touch. In turn, this means that the number of touch points collected for an ink annotation is also higher. A higher sampling rate combined with the higher precision of the format (float) points are stored in contributes to the size of the data in an ink annotation.

The number of touch points reported by iOS while drawing using a touch (finger) is still lower than the number of touch points reported while drawing with Apple Pencil. This is because they’re sampled at a different rate; with a touch sampling rate of 240 hz, Apple Pencil is sampled two times more than a touch.

This makes sense, as one of the uses of Apple Pencil is that of high-fidelity drawings. This kind of use also requires the highest form of precision, which is why Apple Pencil has subpixel precision.

The easiest way to observe this is by adding a gesture recognizer to a view and then trying to draw a dot in the view with Apple Pencil and moving it slightly, which is what we did. The gesture recognizer reports plenty of touch points for even the tiniest shift in the movement of the pencil.

Having many touch points leads to potential hindrances in other aspects of PDF document viewing and annotation. For example, each annotation would comprise of a larger number of touch points. Storing this many points with higher precision means they’ll take more space whenever a document is saved by writing to disk. As a result, the size of the document increases even more as the number of annotations increases.

Additionally, more memory is used while displaying a PDF document, as the document will require more memory to load the annotation touch points for rendering. This increase in the memory usage inadvertently affects the rendering performance by slowing it down when loading a PDF document with a large number of annotations.

All these things clearly demonstrate that there’s room for improvement in the creation of ink annotations. And, of course, improving ink annotation creation will have a positive impact on the overall experience of document viewing and annotating.

Improvements

Instead of looking into a completely different solution, attempting a refactor of how ink annotations are currently created, or using Apple’s PencilKit, we decided to make small improvements. Since our SDK isn’t meant for high-fidelity drawings like many art and illustration SDKs and applications that require precise touch inputs, we had room to make adjustments and not worry about things an illustration app might have to consider, like extreme accuracy and pixel-perfect rendering.

One potential improvement was to optimize the collection of touch points for an ink annotation. All we had to do was only collect relevant touch points and discard redundant points that weren’t needed for a drawing, while still keeping the drawing accurate. “That doesn’t sound too hard,” we thought. Spoiler alert: We were wrong.

We went to the drawing board, quite literally, and started to sketch out the possible ways of optimizing the collection of touch points.

Our first step was to filter out the points that are close to each other. We set a threshold for the distance between points where two points can be considered identical. This resulted in an immediate win in the form of reduced ink annotation size, but we were sure there were even more performance improvements we could squeeze out with some additional optimizations.

The second part of the solution we came up with was to reduce the number of points in a straight line. It involved keeping only the start and the end points of a drawn straight line. We also defined a tolerance within which a point can be said to be on a straight line.

Next, we decided to take the route of trial and error to ensure that we could extract the best reduction from our algorithm with no loss in the fidelity of a drawing. This involved a lot of testing, as we tried many different tolerance levels for both of the steps outlined above to get the results. This was a manual process, as we wanted to get to a result that would seem optimum when the ink annotation was carried out in real time.

In the video below, we’re using Apple Pencil to scribble on a PDF using the ink annotation tool. For every stroke drawn, the number of points fed to the app by the system can be found next to Original, while the actual number of points that comprise the stroke after the above reduction of points can be found next to Reduced. If you look at the numbers, you can see how well our solution works without affecting the fidelity of the annotation.

Conclusion

With the solution outlined above, we were able to reduce the size of ink annotations up to 50 percent. This resulted in improved loading and viewing of a document with a lot of annotations, as well as reduced sync times for Instant.

Apple provides a decent of set of APIs with PencilKit, which can cover most use cases. However, taking the low-level approach of using a gesture recognizer can prove more flexible if you wish to have more control and want to carry out additional processes like the one discussed above.