Understanding Smart Punctuation on iOS: Impact on string handling in apps

The early 20th century saw much of the world adopt the machine that most modern keyboard layouts are based on: the typewriter. Typewriters had quite a few mechanical constraints, such as the force one needed to apply to ensure the characters were correctly printed, their relatively immense weight, and the amount of space every new character occupied. To address the last problem, the keys for “curly” opening and closing quote characters were replaced with a single “straight” quote, freeing up space for two other characters that were deemed to be more important. Modern software doesn’t have these limitations, but many of us developers still use the ambidextrous quotes rather than their typographically correct equivalents, mostly because our keyboards don’t have explicit keys for them. In this post, we’ll take a look at how iOS 11 works around this limitation and how it affects string handling when using UIKit controls.

Smart Punctuation

With iOS 11, Apple introduced Smart Punctuation. By default, this feature automatically converts ambidextrous straight quotes to curly quotes, in addition to converting two hyphens (–) to an em dash (—). In the case of quotes, it also replaces the straight apostrophes and quotes with language-specific glyphs.

Major text input controls in UIKit, like UITextField and UITextView, perform these conversions as the user is inputting text. For quotes specifically, UIKit provides the smartQuotesType property as a part of the UITextInputTraits protocol to control this substitution.

Changes to string handling

For us developers, smart quotes can cause issues in two places: UI tests, and detecting quoted text (in search, for example).

While simply using the smartQuotesType property to disable smart quotes on all input controls might seem like a quick fix, it’s nothing more than a bandage that quickly falls off: It is easily circumvented by pasting in text that contains smart quotes.

In our PDF Viewer app, we enabled users to search for text across all their documents by leveraging the Indexed Full-Text Search component of the PSPDFKit framework. When searching, we tokenized the input text using the Porter-Stemming technique, in order to provide more results. Since this was at the cost of result accuracy, users could enclose their search terms in straight double-quote characters and only get results that matched the input exactly.

We added a small extension on Swift’s String type to check if the receiver was enclosed in quotes. This was our implementation:

extension String {
    var isEnclosedInDoubleQuotes: Bool {
        return characters.first == "\"" && characters.last == "\""
    }
}

This implementation was pretty naïve, as not only did it check just for the straight quotes, but it also didn’t consider the variety in quote styles across different languages. While we evaluated certain ways to fix this (as described below) we ended up not having to fix it — our app switched from a custom file browser UI to using iOS 11’s native UIDocumentBrowserViewController, which meant we relinquished control over search, and therefore, the text field. However, had we had to fix it on our own, we would have used the following two steps: correctly detecting quoted text, and replacing smart quotes with straight quotes.

Correctly detecting quoted text

To do this, you’d need to account for the keyboard’s current language using UITextInputMode.primaryLanguage, and then check for the appropriate glyphs at the start and end of the string returned from the input control. This can easily become very complex depending upon the languages you support, and it is a bit of a balancing act between how typographically accurate you wish to be and how convenient you want the user input to be.

Frank Rausch gave a talk about building better apps with good typography. In this talk, he touched upon the differences between the types of dashes, and, of course, quotes. He also pointed us to his excellent open source library, Typographizer. Given that it is available to use under the lenient MIT license, you can look at the various cases that need to be handled to properly detect quotes. Using this information to develop a better version of the isEnclosedInDoubleQuotes computed property above is left as an exercise for the reader.

Replacing smart quotes with straight quotes

In some cases, your user-input text will be more than just user facing; it can have meaning for other parts of your code. Consider, for instance, an SSH app like Prompt. If the app uses a UITextField for input, and if you don’t set the smartQuotesType property to UITextSmartQuotesTypeNo, then a command like find . -name 'Hello World.txt' will get turned into find . -name ‘Hello World.txt’, resulting in a syntax error. In some cases, this replacement may not even cause an error, but instead cause unexpected behavior, which is far worse.

A first step, if using a UITextInputTraits conforming control, would be to set smartQuotesType to UITextSmartQuotesTypeNo, and set smartDashesType to UITextSmartDashesTypeNo. However, as mentioned above, this doesn’t work around the issue of the user pasting in text containing these characters. This is an issue across all iOS versions, not just iOS 11.

To truly sanitize your input, you would then need to check the input string whenever it changed and replace the special characters with their straight-quote equivalents. This isn’t a solution that will always be appropriate, though; the user might intentionally input specific characters, and there’s no easy way to tell what the user means without employing heuristics, which, by definition, will not always be accurate.

In a slightly anti-climactic conclusion, it appears that the best solution is dependent upon what your app does, what the particular input in question does, and, most importantly, what the user wants to do.

FAQ

Here are a few frequently asked questions about Smart Punctuation.

What is Smart Punctuation on iOS?

Smart Punctuation on iOS automatically converts straight quotes to curly quotes and double hyphens to em dashes in text input fields.

How does Smart Punctuation affect string handling in apps?

It can cause issues in UI tests and when detecting quoted text, as the automatically substituted characters may not match the expected straight quotes.

Can Smart Punctuation be disabled in iOS apps?

Yes, you can disable it by setting the smartQuotesType and smartDashesType properties to UITextSmartQuotesTypeNo and UITextSmartDashesTypeNo, respectively.

How can I detect if a string is enclosed in quotes?

You can check for language-specific quote characters at the start and end of a string, taking into account the keyboard’s current language.

Why is it important to replace smart quotes with straight quotes in some apps?

Smart quotes can cause syntax errors or unexpected behavior in commands or code snippets, as they differ from standard straight quotes.

What should I do if pasted text contains smart quotes?

You should sanitize the input by replacing smart quotes with straight quotes to ensure consistent behavior in your app.