Investigating a Dynamic Linking Crash with Xcode 16

When Xcode 16 was released as a beta build at WWDC24, we were quick to download it and start testing our product with iOS 18. However, when running our project for the first time, we noticed a crash on the launch of our example app when initializing PSPDFKit, and it was related to dynamic linking of symbols. The origin of this issue and the fix for this weren’t immediately obvious, so this blog post will go into details of how we debugged the issue and came up with a solution.

Identifying the Issue

The crash we saw happened when initializing PSPDFKit, which caused an EXC_BAD_ACCESS error message on Xcode 16. More specifically, this crash was happening for customers using Xcode 16 with already released versions of PSPDFKit, which is distributed as a binary framework. We even had a customer contact us on support with the same issue shortly after we discovered it during testing.

As such, it was all the more important for us to fix the crash. This is because it wasn’t related to our framework built with Xcode 16; the issue also happened when our framework was built with Xcode 15. It’s great to see our customers eager to adopt the latest technologies, and we were keen to unblock them.

We discovered that calling sqlite3_threadsafe is what caused the issue. But for some more background on this, we have to dive into why we’re calling this function in the first place.

PSPDFKit for iOS uses SQLite internally for various features, like full-text search. By default, we use the SQLite library the system provides. However, it’s also possible for customers to provide their own version of SQLite if they’re linking a custom SQLite library in their project — in which case, PSPDFKit for iOS uses this library instead. Since we can’t control which SQLite library is used, we perform some checks to verify that all the symbols PSPDFKit uses from SQLite are available in the provided library.

Note that we’re using C in our codebase to work with SQLite; this is a mature part of our codebase that has been proven to be reliable. One of the verification steps in place is making sure all the SQLite symbols we load come from the same dynamic library. If that isn’t the case and there are symbols from another library, we abort the process and we don’t link SQLite. This is accomplished using code along the lines of the following:

bool pspdf_sqlite_dynamic_linking_check(const void *handle, const char *method, Dl_info *last_info) {
   void *address = dlsym(handle, method);
   Dl_info info;
   dladdr(address, &info);
   if (last_info->dli_fname && strcmp(info.dli_fname, last_info->dli_fname) != 0)) {
      LOGWARN("shared object for %s (%s) doesn't match the one from the previous method (%s)", method, info.dli_fname, last_info->dli_fname);
      return false;
   }
   *last_info = info;
   return true
}

void *handle = dlopen(sqlite_path, 0);
Dl_info last_info = {0};
if (!pspdf_sqlite_dynamic_linking_check(handle, "sqlite3_config", &last_info)) { return false }
if (!pspdf_sqlite_dynamic_linking_check(handle, "sqlite3_close", &last_info)) { return false }
if (!pspdf_sqlite_dynamic_linking_check(handle, "sqlite3_threadsafe", &last_info)) { return false }
...

In the code above, sqlite_path is either libsqlite3.dylib or the path to the custom SQLite library the customer provided.

We check whether the function to be loaded comes from the same library as the previous one, which is done by seeing if info.dli_fname is the same as last_info->dli_fname. dli_fname is the path name of the object that contains the address of the symbol we just loaded into memory.

If the check fails, we return false and stop loading the rest of the SQLite methods using dlsym, in which case only a subset of functions are loaded. When using the libsqlite3.dylib system library with Xcode 15, everything works as expected, and all symbols are loaded from the same library. However, after switching to Xcode 16, we noticed a crash related to this logic.

More specifically, after calling and searching for SQLite symbols using dlsym(handle, "sqlite3_close"), we observed an unexpected value for Dl_info.dli_fname. Instead of pointing to libsqlite3.dylib, the reported path is /Library/Developer/CoreSimulator/Volumes/iOS_22A5282m/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 18.0.simruntime/Contents/Resources/RuntimeRoot/usr/lib/libRPAC.dylib. This was unexpected and caused our validation check that all symbols need to come from the same library to fail.

Also, not all symbols were reporting the libRPAC.dylib path; only some of the SQLite-related methods did. And since sqlite3_threadsafe is the first method we call after loading SQLite has been aborted, resulting in the function not being loaded, we get the crash mentioned above.

Investigating the Problem

Once we narrowed down what caused the crash, we could begin investigating why it happened. The incorrect path including libRPAC.dylib suggested that the dynamic linker was resolving symbols from the wrong library. This behavior is inconsistent with previous versions of Xcode, where all the SQLite-related methods we were loading were coming from libsqlite3.dylib.

What this means is that there was a change in Xcode 16 related to the libRPAC library. Upon inspecting the libRPAC.dylib binary using nm -g, we discovered that it now includes some SQLite-related symbols. This wasn’t the case in Xcode 15 (with the iOS 17 SDK).

Figuring Out a Solution

Since we noticed this right after Xcode 16 beta 1 was released, we still had time to book a WWDC lab, where we received some pointers on how to debug the situation and figure out what causes it. It was motivating to have the opportunity to dig into this with engineers at Apple and see them at work. Labs are a fantastic part of WWDC, and I recommend booking them if you’re ever able to.

Some of the tips we got were to add environment variables to the scheme in Xcode to figure out why libRPAC is loaded and why the addresses aren’t coming from libsqlite. For this, we tried setting DYLD_PRINT_SEGMENTS and DYLD_PRINT_APIS to 1. Doing so logs events related to dynamic linking to the console.

Then we set another environment variable, DYLD_PRINT_LINKS_WITH libRPAC.dylib, which logs to the console whenever the specified dynamic library is loaded. In our case, this happened right when launching the app, so it looked like libRPAC was something Xcode inserted.

While debugging this together with Apple engineers, we discovered libRPAC is interposing SQLite methods. But why is libRPAC even loaded in the first place? To figure this out, we disabled various settings in the Options and Diagnostics tabs in scheme settings until we found a setting that caused libRPAC to be loaded. We found out that the culprit was the Thread Performance Checker tool, which is enabled by default in the scheme settings in Xcode. The Thread Performance Checker tool loads libRPAC at app startup, and when disabling the tool, we could no longer reproduce the crash.

With the Thread Performance Checker tool, libRPAC apparently does some analyzing behind the scenes for SQLite symbols. With that knowledge, we immediately recommend the customer who reported this to disable the Thread Performance Checker tool. However, this isn’t a long-term solution, as we don’t want to recommend disabling useful features. So we also adjusted our checks to allow libRPAC.dylib in addition to libsqlite3.dylib. This only requires customers to update the PSPDFKit version to avoid this crash, while still being able to keep the Thread Performance Checker tool enabled.

Conclusion

We already shipped this fix with PSPDFKit 13.7 for iOS, which now won’t crash on Xcode 16.

Dynamic linking issues can be tricky to diagnose, and the reason for a change like this might not be immediately obvious. But by investigating the path discrepancies and symbol inclusion, we were able to resolve the issue and make sure running with the newest Xcode beta builds won’t crash.