How to Identify Browser Bugs
Nowadays, browsers are among the preferred targets for application development. Web technologies allow us to easily offer cross-platform experiences and target both desktop and mobile devices within a single codebase.
Web browsers are far from perfect though. As more and more features are shipped in browsers and new APIs allow developers to build new experiences, there’s more room for bugs to sneak in in unexpected ways.
When our web applications don’t behave like we expect, how do we know if we made a mistake when writing our markup, styles, or business logic? Or how do we know if it turns out to be something that isn’t playing out as it should on the underlying rendering engine or JavaScript engine?
Reproducing Across Browsers
Consider this scenario: You receive a bug report from one of your users, so you switch into detective mode and start digging around, expecting to identify the culprit within your codebase. After a while, you’re finally able to reproduce the buggy behavior. But then, how to tell whether it’s your code that’s the issue or a browser bug that’s causing the problem?
The first step is to try and reproduce the behavior across multiple browsers. Please be aware that some browsers share the same underlying engine, so for instance, if you try out Google Chrome, Opera, and Microsoft Edge — all of which are based on Chromium — it’s likely the behavior will be consistent.
A better combination of browsers to try would be Google Chrome, Safari, and Firefox, since all of them use different underlying rendering and JavaScript engines. Additionally, testing across platforms and devices might shed some light on where the issue exists.
If the behavior is only reproducible on some browsers but not on others, chances are you’ve stumbled upon a browser bug.
Isolating the Problematic Behavior
At this point, it’s valuable to work on a small test case, isolating the DOM elements, CSS rules, and JavaScript code relevant to the problematic interaction. The less code needed to trigger the wrong behavior, the better.
For this, start simplifying the interaction by extracting the code piece by piece and see if the bug is reproduced. If it’s not reproducible yet, try adding additional elements until you trigger it.
If you’re unable to trigger the bug even after adding multiple parts of the interaction, it’s time to take a deep look into your architecture. This might be a sign of multiple parts being tightly coupled, considering that it’s hard to find out all the relevant code paths involved in an isolated interaction. Globals, shared mutable state, and high indirection are usually tricky and can cause these maintainability and debugging difficulties.
Once you have a small enough test case ready, it’s time to go through different browsers again and check if the behavior is still inconsistent across them. If it is, you’ve identified some inconsistent behavior.
The test case is useful for two things:
-
You have an isolated and easy-to-reason-about base code that you can use to find a (hopefully temporary) workaround.
-
You can use the test case to file a bug report in the bug-tracking system of each browser. This helps make it easier for developers to reproduce the bug and ship a fix more quickly.
Here’s an example of a recent test case we made for a browser bug we identified. The relevant parts are all on 36 lines of JavaScript code and 26 lines of CSS code.
This particular bug occurs when scrolling using the arrow keys. On Chrome 86 or Safari 13.1, try clicking on the yellow box and then scroll down solely using the down arrow key. Notice how when you reach the blue box, you’re unable to continue scrolling using the keyboard.
Determining Expected Behavior
Before going ahead and opening a bug report, you need to identify exactly what the expected behavior is.
If the problem is only present on a single browser, then it’s highly likely that the expected outcome is for it to work like it does on other browsers, and if you end up filing a new bug, you can make a note about how it’s working correctly on other implementations.
Usually, a bug is acted upon much more quickly if it’s a regression. To find out if this is the case, you can rely on tools like BrowserStack to try an older version of the browser. If it used to work OK, try to find the latest working version and make a note of it.
However, even in cases where all other browsers are working in a way that doesn’t match what you expect, perhaps the presumably wrong browser is actually the one that works like it’s supposed to.
The way to confirm the expected outcome is by taking a look at the underlying specification that defines the feature in question. By contrasting the different behaviors with what the specification states, you’ll have conclusive evidence that the identified behavior is indeed a bug. Again, if you end up needing to file a bug, please also include the reference to the relevant specification section.
Chromium issue #1016974 is an example of a bug we identified and reported. Notice how it includes information about failing versions and a reference to the underlying specification — the Pointer Events specification in this case — regarding the correct behavior.
Filing the Bug
Once you’re certain you’re dealing with a browser bug, the next step is to go to the browser’s bug tracker. Here’s the list of the official bug trackers for the main browser engines:
All of these trackers offer built-in search engines for looking into existing bugs. There are more chances an issue is fixed promptly if multiple users are affected by the same issue, so if you’re lucky enough, someone else already stumbled upon the same problem and there’s an existing bug report about it.
If that’s the case, you can contribute to the discussion, providing your feedback or additional insights into how the bug behaves. Perhaps your test case triggers the wrong behavior in a slightly different way, in which case it’ll be valuable to share this with others so that the fix for it is hopefully found more quickly.
One example of this is what we observed on Chromium issue #989643. We noticed that official documentation and online resources mentioned icons associated with the Memory tab on Chrome DevTools. However, these icons weren’t visible on macOS. But as it turns out, an issue already existed!
If you don’t find a similar bug report, it’s time to open a new one. It takes just a few minutes since you’ve already done the hardest part of the job, which is coming up with a good reproduction case.
Here are some key things your bug report should contain:
-
A brief but descriptive title
-
The test case
-
Steps to reproduce the issue
-
The expected behavior
-
A list of affected browsers with their version(s) and platform(s)
-
Whether or not it used to work on an older version — if it’s a regression, try to specify the latest working version number
-
A list of browsers in which it works as expected
If you want to learn more about how to file a good browser bug report, take a look at this excellent article by Robert Nyman and Pete LePage.
After a bug report is submitted, it might take a while until others see it. Usually, a member of the browser’s development team tries to reproduce the issue and starts a discussion around whether or not it’s really a bug. They might ask for additional information, or some discussion might start around the best way to solve the issue and what changes to the internal implementation might be needed.
It’s common for multiple members of the browser development team to jump in until someone is assigned to solve it. Afterward, once a fix is merged, the next release that will include the fix is announced on the thread. At this point, you can download a build containing the fix and verify that a bug is no longer present.
Earlier this year, PSPDFKit for Web was affected by a regression introduced on Chrome 83 that broke transparent text selection. We wrote about it in our Text Selection Regression in Chrome 83 blog post, and fortunately, the underlying bug has since been fixed.
Going back to our actual test case of the failed scrolling using the arrow keys that we introduced earlier, we went ahead and opened Chromium issue #1138529 about it. A few hours after the bug report was opened, a member of the Chromium team was able to confirm it and mark it as available to be picked up and fixed.
Finding a Workaround
So far, you have a reproducible test case and have filed a bug report hoping for a fix in the near future. In the meantime, you might need to be creative and come up with a workaround until the fix arrives. Whether or not a workaround is possible depends upon the complexity of the underlying bug, but usually you can come up with something.
One example of a bug that was impossible to work around was Chromium issue #1013635. As you can see from the discussion, multiple applications were affected by it, but fortunately a fix shipped a few releases later.
Thanks to the very same reproduction test case shown above, you can experiment and try different possible hacks without dealing with the complexity of your main application, which involves more layers of complexity that might make debugging more difficult.
An ideal workflow would involve modifying the test case so that the bug is no longer present, and then going back to the real application and applying the same fix there.
Please make sure to document the need for the hack properly and include a link to the associated browser bug. Comments are the perfect solution for this.
When coming back to your code in the future, verify if the fix has shipped and if you no longer need to support the previously affected browser version. If that’s the case, please remove your workaround and verify that the behavior is consistent across all the browsers your application supports, and enjoy the size reduction on the code you ship to browsers. 🙂
Another recommendation is to only apply the workarounds on the browsers that aren’t working as expected. That way, unaffected browsers can still rely on the default behavior. For this, there are many techniques available. You can use a dependency like Bowser, parse the navigator.userAgent string with something like what this Stack Overflow answer recommends, or rely on duck typing like this other Stack Overflow answer describes.
Conclusion
This post outlined some strategies for finding and reporting browser bugs. We encourage everyone to report bugs and keep making the web a better platform.
At PSPDFKit, filing bug reports is an essential part of our culture. You might be interested in reading our Writing Good Bug Reports blog post, which outlines additional tips. Although it’s written from the context of iOS bug reports, the same principles can be applied to the web and other platforms.
