Supporting Pointer Interactions

With iOS 13.4, Apple added support for trackpad and mouse devices for iPad. While these devices have been supported since iOS 13, up until this point, that support was purely an accessibility feature, and it lacked the APIs to adjust apps for it.

However, this week’s release added new APIs for developers: UIPointerInteraction, UIPointerEffect, UIPointerShape, UIPointerRegion, UIPointerStyle, and more.

Enabling Pointer Support

Apple uses a compatibility mode to simulate touches from indirect input devices. The system creates synthetic touches for gestures, so legacy code will mostly work just fine, and by default there’s no public way to detect if a touch has been created from a tap or an input event. As a primer on how pointers should work on iPad, I suggest reading the new pointer section in Apple’s Human Interface Guidelines.

To indicate your app is optimized for indirect input mechanisms, declare UIApplicationSupportsIndirectInputEvents in your Info.plist. With this key enabled, touches will have a new type of UITouch.TouchType.indirectPointer, and custom touch handling and gesture recognizers will need to be updated.

Be sure to read the related discussion on Apple’s documentation to understand all the side effects. For example, scroll or gesture events can now become active without touches. As a result, using an API like location(ofTouch:in:) will throw an exception. This can be prevented by checking that numberOfTouches is not zero.

However, there is quite a bit of low-hanging fruit to help you improve pointer support without going all in.

UIButton

UIKit automatically enables pointer handling for UIBarButtonItem, UISegmentedControl, UIMenuController, and various other controls (the latter isn’t documented, so there are probably more). When you go over one of these controls, the pointer automatically morphs into an area. For flexibility, our annotation toolbar uses UIButton instead of UIBarButtonItem, so we don’t get any special pointer optimization by default.

Apple added an unusually large number of new APIs in the stable release of iPadOS 13.4. One property you might have overlooked is isPointerInteractionEnabled, which is exactly what we want here:

let button = UIButton(...)
if #available(iOS 13.4, *) {
    button.isPointerInteractionEnabled = true
}

That’s all we have to change to get great pointer feedback for our buttons.

Alternatively, you can also set a closure — pointerStyleProvider — for custom shapes and effects.

Other Views

For other views, making them react to pointer events is a bit more work: We need to use the new UIPointerInteraction API. The API is based on UIInteraction — the same protocol you already know from the drag-and-drop additions to iOS 11. A UIPointerInteraction can be added to any view, and it then calls the UIPointerInteractionDelegate delegate.

If you’re using UIView or UIControl for buttons, you need to add a UIPointerInteraction and implement a delegate method:

class PointerControl: UIControl {

    override init(frame: CGRect) {
        super.init(frame: frame)
        if #available(iOS 13.4, *) {
            enablePointerInteraction()
        }
    }
    required init?(coder: NSCoder) {
        super.init(coder: coder)
        if #available(iOS 13.4, *) {
            enablePointerInteraction()
        }
    }
}

@available(iOS 13.4, *)
extension PointerControl: UIPointerInteractionDelegate {

    func enablePointerInteraction() {
        self.addInteraction(UIPointerInteraction(delegate: self))
    }

    // This isn't even needed — just indicating what the default does!
    func pointerInteraction(_ interaction: UIPointerInteraction, regionFor request: UIPointerRegionRequest, defaultRegion: UIPointerRegion) -> UIPointerRegion? {
        return defaultRegion
    }

    func pointerInteraction(_ interaction: UIPointerInteraction, styleFor region: UIPointerRegion) -> UIPointerStyle? {
        return UIPointerStyle(effect: .automatic(.init(view: self)))
    }
}

Supporting Custom Cursors

With UIPointerInteraction, it’s easy to provide custom cursors for various elements. Consider our annotation selection logic. Anyone who has ever used Photoshop or a similar application expects that the pointers will change into resize handles when hovering over the resize knobs.

This is quite easy to achieve. First, create an interaction and add it to your view:

self.addInteraction(UIPointerInteraction(delegate: self))

Then, implement the region delegate. This is called every time the cursor moves, so you can return different cursors for different regions in your view. If you return a UIPointerRegion object, UIKit will call the next delegate:

func pointerInteraction(_ interaction: UIPointerInteraction, regionFor request: UIPointerRegionRequest, defaultRegion: UIPointerRegion) -> UIPointerRegion? {
    // Custom logic that returns a struct with pointer details and a subrect for the view.
    let cursorMode = cursorAt(point: request.location)
    return UIPointerRegion(rect: cursorMode.cursorArea)
}

Once we return a region, UIKit will call the second delegate and ask for the pointer style. This is simplified code to make it easier to understand:

func pointerInteraction(_ interaction: UIPointerInteraction, styleFor region: UIPointerRegion) -> UIPointerStyle? {
    let cursor = // We store our cursor data in a class property and refetch it here.
    var pointerShape = self.view.convert(cursorMode.cursorArea, to: self.view.window?.coordinateSpace)
    let path = PointerBezierPath(cursor.cursorMode, pointerShape.size, cursor.additionalRotation);
    return UIPointerStyle(shape: UIPointerShape.path(path))
}

Custom Text Handling

Text that is displayed via UITextView or UITextField is automatically recognized by the cursor, which switches into a beam cursor. In our SDK, we needed more flexibility in font handling, so text selection is custom. The cursor can be changed via the UIPointerShape.verticalBeam enum.

API-wise, this is very cool, as UIPointerShape is a class in Objective-C and an enum with an associated value in Swift. Apple goes to great lengths to make UIKit convenient for Swift users here!

Text selection works differently, depending on the input created: Use touches and you get selection handles and some delay to allow scrolling. With an indirect input device, text selection is immediate and doesn’t require handles, as the input device is more accurate by default. Our custom logic nicely handles both cases.

Tips and Tricks

A Better Way to Check for Pointer Interaction Support

If you have a widely used app in the App Store, you’ve seen some really weird crashes. Adding pointer support and only protecting it with an availability check for iPadOS 13.4 will crash for everyone who hasn’t updated from the beta yet. And while you shouldn’t optimize for people using beta releases, a simple class check will go a long way in reducing hits in the “weird crashes” category:

static BOOL PSPDFSupportsPointerInteraction() {
    if (@available(iOS 13.4, *)) { return NSClassFromString(@"UIPointerInteraction") != nil; } else { return NO; }
}

Implementing Pointers When CI Is Still on Xcode 11.3

You might need some time to update CI to the latest Xcode release. But maybe you still want to start implementing pointer interactions right away. We added a support file named PSPDFPointerInteractionSupport.h, and it contains all the header declarations necessary to achieve this. Accessing the classes doesn’t work at link time, but you can write rather readable code using the following trick:

- (nullable UIPointerRegion *)pointerInteraction:(UIPointerInteraction *)pointerInteraction regionForRequest:(UIPointerRegionRequest *)request defaultRegion:(UIPointerRegion *)defaultRegion {
    let cursorDetails = [self cursorModeForGestureState:UIGestureRecognizerStateChanged atPoint:request.location];
    _cursorDetails = cursorDetails;
    if (cursorDetails.cursorMode != PSPDFCursorModeArrow) {
        return [NSClassFromString(@"UIPointerRegion") regionWithRect:cursorDetails.cursorArea identifier:@"cursor"];
    }
    return nil;
}

Differentiating between Pointer Events and Touches without UIApplicationSupportsIndirectInputEvents

While there’s no public way to detect if a tap has been created on a pointer event, it might still be useful to check for this while debugging. UITouch has a private method, _isPointerTouch, that indicates this:

let isPointerTouch = button.value(forKey: "_isPointerTouch") as? Bool ?? false

If you’re wondering how to find secrets like this, here’s my Twitter thread on implementing pointer interactions before Xcode 11.4 was released.

Don’t Change Cursors Rapidly

I’ve experimented quite a bit with the rotation cursor and also tried to rotate it dynamically, as the user rotates. While UIKit generally makes really nice path animations, this doesn’t look so great.

A better way is to remember the initial rotation and only update it once the user stops rotating.

Disable Pointer Animations

Users can disable pointer animations in Apple’s Accessibility settings.

Here’s how our UI looks with pointer animations disabled. The cursor still morphs into different shapes, but it doesn’t disappear.

Conclusion

We’ve only spent a few days on pointer support so far and already made huge progress — Apple clearly released well-designed APIs here. You can soon try for yourself in PSPDFKit 9.3 for iOS and in our free PDF Viewer app. This is just the beginning, and we have quite a few more ideas to further improve trackpad and mouse input. We’re also excited to see what Apple and the community will come up with in future app updates.