Yesterday I became the owner of Gecko’s NPAPI plugin module. Johnny Stenback, the previous owner, remains on as a peer. Given that change, now seems like a good time for an update on plugin module priorities. My current priorities are related to testing, accessibility, and code cleanup.

Automated testing for NPAPI plugin functionality has been highly requested for a while. In bug 386676 I posted a patch that allows for plugin mochitests. The first test gives us coverage for a basic plugin load plus basic npruntime calls into and out of a plugin. As soon as that lands I hope we’ll be adding many more plugin tests and expanding our testing capabilities.

I am working on solutions to two major accessibility bugs, as discussed in this post. We need to allow plugins to reject keyboard events so keyboard commands for things like close-window can work while a plugin has focus, and we need to allow for changing focus from a plugin to other content via the keyboard. Work is slow but steady, things should speed up as other things like testing fall into place.

Gecko’s plugin module is in need of some general TLC. This includes code formatting, dead code removal, renaming/re-factoring for readability, general ugly/outdated code auditing, and sample modernization. I landed some of that already and I’m hoping to land a bunch more over the coming months.

I’m not ready to write about longer-term priorities yet, things like out-of-process plugins. We are thinking about them though, I’ll have more information in January or February. In the mean time I’m looking forward to making a lot of progress on things that will make the bigger changes easier to digest.

I’ve added two new peers to the “Widget - Mac” module (Cocoa widgets), Steven Michaud and Markus Stange. They have both done great things for the module and working with them is a pleasure. Steven has contributed a huge number of fixes for tough bugs over the past two years, focusing on event handling, plugin interaction, and crash debugging. Markus joined us earlier this year and has gotten up to speed very quickly. He has contributed many fixes focusing on theme and OS integration issues.

Thanks Steven and Markus!

We’ve started working on Gecko 1.9.1 for Mac OS X. It is early in the development cycle and things could change, but I want to give people an idea of what we’re planning on doing as of now.

Aside from the usual bug squashing, we’re going to focus on minimizing Carbon usage and getting ready for 64-bit. Gecko 1.9.0 is generally Cocoa-based but it still contains a modest amount of Carbon and other code that is not 64-bit-ready. We’re probably not going to be Carbon-free or 64-bit-ready for the Gecko 1.9.1 release, but we can make a lot of progress.

• I’m adding support for NPAPI plugin event model negotiation and the Cocoa event model in bug 435041. This will allow for Carbon-free plugins and is major step towards 64-bit Gecko on Mac OS X.
• I’m working on new file system interaction code for Mac OS X in bug 438694. The goal of this work is modern and clean 64-bit-ready code that uses supported APIs.
• I’m also hoping to rewrite our print dialog implementation in Cocoa. It is one of the few components that are still completely Carbon-based.

We’ll also be doing some long-overdue general cleanup and performance work.

• I have rewritten much of our native menu code in bug 433952. We’re doing this to improve code size, code clarity and run-time speed. The new implementation is completely decomtaminated, ~700 lines of code lighter, better organized and much easier to understand. In the future we will be able to make changes and fix bugs much faster. It will also be easier to port this new implementation to other platforms that want native menus, such as mobile GTK.
• Our child view and top-level window code is very complicated - some of that we can’t help, but there are steps we could take to make it easier to understand and work with. I’m hoping to do some re-factoring and documentation work for Gecko 1.9.1.
• We’re planning to expand our widget testing framework to include things like focus testing and more advanced key handling tests.

There will probably not be many new Mac-specific platform features added to Gecko 1.9.1, but there are at least a couple of nice ones on the way.

• Atul Varma is adding support for HTML data on the system clipboard in bug 428096.
• James Bunton and others have worked together in bug 125995 to add support for taking proxy settings from Mac OS X network preferences.

Firefox 3 will be released soon (get the RC here). While the release contains a huge number of new features and performance improvements for all platforms, it is particularly significant for Mac OS X users. We rewrote most of the Mac OS X code that was behind Firefox 2 in order to benefit from modern Apple technologies and fix long-standing bugs. Once you try it I think you’ll agree that the results are astounding. I’d like to explain what exactly we did in this rewrite, how Firefox 3 for Mac OS X is different “under the hood.”

Before I start, I need to explain Gecko vs. Firefox for anyone who doesn’t already know. Gecko is the engine behind Firefox. It provides the capabilities that we use to build Firefox. Under the umbrella of Mozilla, Gecko is actually a much bigger project in technical terms than Firefox is. For example, there is much more Gecko code than there is code specific to Firefox, which is an application we built on top of Gecko. Firefox and Gecko have different version numbers - Firefox 2 uses Gecko 1.8.1 and Firefox 3 uses Gecko 1.9. This post is about changes we made in Gecko 1.9, the engine behind Firefox 3.

The biggest change is that Gecko 1.9 is based on Cocoa instead of Carbon on Mac OS X. There has always been a lot of confusion about what this means, particularly since Gecko 1.9 also happens to include Aqua form controls. It may seem strange to some, but Gecko’s new Cocoa underpinnings and its Aqua form controls are almost completely unrelated.

There are only 2 types of Cocoa objects at the heart of Gecko 1.9 on Mac OS X (let’s forget about the menu bar at the top of the screen for now). Cocoa’s NSWindow allows us to make and control a window on the screen and Cocoa’s NSView allows us to draw things into a window. Those two objects also allow us to get most of the events we need to know about from the operating system (such as key and mouse events). We do not use actual Cocoa buttons or any other Cocoa controls within any Gecko 1.9 windows. The context menus, dropdown menus, the toolbar, the search bar, the buttons and text fields within web pages - they are not actual Cocoa controls. For example, instead of using actual Cocoa buttons for “Submit” buttons we just draw the image of an Aqua “Submit” button into an NSView, one of the basic Cocoa objects we use. Gecko 1.9 has Aqua form controls because we now draw images of Aqua form controls when appropriate, not because we use actual Cocoa controls. The reason we don’t use actual Cocoa controls isn’t because we are lazy or we can’t figure out how to use them or because we are constrained by our cross-platform requirements - Apple’s WebKit doesn’t use actual Cocoa controls for things like “Submit” buttons or popup buttons in web pages either, at least not the last time I checked. IE for Windows is in the same boat. The reason Gecko 1.9 doesn’t use Cocoa controls is for the sake of flexibility - form control behavior and appearance can be changed significantly via HTML, CSS, and JavaScript. Actual Cocoa controls are simply not flexible enough to do all of the things that people want to be able to do with controls on the web.

So if we didn’t get Aqua form controls out of the deal why did we do the Cocoa rewrite? First of all, Apple has deprecated much of Carbon already¹ and has made it clear that Cocoa is the future for Mac OS X applications. Apple is investing heavily in Cocoa, which benefits us if we use Cocoa. Cocoa also gives us access to great features that would be more difficult or impossible to take advantage of through Carbon. For example, with Cocoa we were able to relatively easily draw using CoreGraphics instead of the ancient Quickdraw API (more on this later). Secondly, the Cocoa way of doing things matches up with the Gecko way of doing things better than Carbon did. Our Cocoa code is easier to understand and maintain because of this². This will result in faster development and fewer bugs in the long run. In fact, we actually added more capabilities to Firefox 3’s Gecko 1.9 (such as transparent windows, unified toolbar windows, and an alert service) than we have ever added in any release of Gecko since Firefox 1.0. This is in addition to doing a huge amount of work just to re-implement many of the features that Gecko 1.8.1 had already implemented in Carbon!

Another major under-the-hood change in Gecko 1.9 for Mac OS X is drawing via CoreGraphics and ATSUI instead of Quickdraw. Like much of Carbon, Quickdraw is deprecated and does not exist in 64-bit Mac OS X. Gecko 1.9 uses the Cairo library on all platforms, and Cairo draws with CoreGraphics and ATSUI on Mac OS X. The CoreGraphics drawing API is modern and hardware accelerated, a huge improvement over Quickdraw in terms of speed, capabilities, and code clarity. In addition to using CoreGraphics ourselves, we have made it possible for plugins to use CoreGraphics via NPAPI Drawing Models³. Flash is already prepared to take advantage of this new capability in Gecko 1.9! As for text, using ATSUI allowed us to improve our text kerning and ligature capabilities. It also gives us better glyph cluster positioning, which is good for Indic languages and languages that use exotic combining marks.

I hope this helps to shed some light on why Firefox 3 is a particularly significant upgrade for Mac OS X users. I’m really proud of what we accomplished with this release, it was ambitious and things worked out well. Kinks remain as with all major revisions, we’ll be addressing those quickly in minor updates to Firefox 3.

==

¹ While the deprecated Carbon API is still available for 32-bit applications like Firefox 3, it simply won’t be available to 64-bit applications. Firefox 3 for Mac OS X will not be available in 64-bit but we’re preparing for 64-bit by removing code that won’t work in 64-bit.
² It might be true that on the whole our Cocoa implementation is more complex than our Carbon implementation was, but that is because our Cocoa implementation does far more than our Carbon implementation ever did. The Cocoa code is still more readable and maintainable.
³ See the NPAPI Drawing Models spec. http://wiki.mozilla.org/Mac:NPAPI_Drawing_Models

If you’ve participated in or followed Mozilla’s Firefox 3 development over the past month you’re probably aware that we had a bunch of issues with key handling come up at the very end of our development cycle. In the interest of giving others an opportunity to learn from our mistakes and to generally communicate what happened, I’ve agreed to write up a summary of the development team’s postmortem discussion about our recent key issues.

“Key hell” started when I fixed bug 398514, a significant rewrite of our key event flow, and Masayuki Nakano started fixing a major bug concerning key command event mapping on international keyboard layouts (more on Masayuki’s fix later). Prior to my patch we let Mac OS X match keyboard commands to native menu items and then executed associated DOM command nodes based on the operating system matching. The up side of that is that the operating system does all the mapping of key commands to their menu items, the down side is that nothing but the command node associated with the menu item ever sees the event. After my patch we ignored the operating system’s invocation of native menu items in favor of allowing the key event to flow normally through Gecko. This fixed a lot of important bugs, I can’t believe we got away with pigeon-holing key equiv events for so long. The problem is that we were now in charge of mapping key events to their commands. For US English this is pretty simple mapping and it works without special treatment, I didn’t notice any problems because I use US English. When I tested with other keyboard layouts I tested text in text fields - I didn’t test many keyboard commands. My bad #1, but I wasn’t aware of the fact that key commands have certain types of complex mappings under different keyboard layouts and circumstances.

Coincidentally, this problem came up for different reasons on Windows and Linux right before I exposed it on Mac OS X. Masayuki had already started working on a fix because of Windows and Linux. By the time we had sorted through bug reports and figured out what was going on with non-US-English keyboard commands on Mac OS X Masayuki was half way to a fix. This ordeal would have been much worse without that stroke of good luck. Masayuki is a talented guy that understands far more about international keyboard layouts and input than I do, I was pretty happy he was already on the problem.

Masayuki’s patch(es) attempted to solve a very complex problem. The problem has a huge number of edge cases under different keyboard layouts and the process of finding and fixing cases that we hadn’t covered yet dragged out until today. I don’t think any particular engineer is to blame for this, the extended timeline for fixing regressions was the result of late detection and a lack of automated test coverage.

We should have discovered this problem much earlier than we did. I suspect that we finally discovered it as a result of heavy beta usage, especially among international users. User testing is great but it is a perk - not something to be relied upon. We also would have found out about this earlier had I committed my major patch for bug 398514 earlier. We had good reasons for making that change so late, but having good reasons doesn’t shield us from the consequences. At best it just makes the consequences easier to swallow.

The other major factor contributing to this debacle was a lack of automated testing until too late. With a decent test suite we could have found out about most of this much earlier. Even after we found out about the problem it took us too long to get tests in place to aid in avoiding regressions in the fix process. Eventually Roc wrote up a great test system (based on synthesized native events) and that has helped a lot. We’ve since added a bunch more tests to his original set and we’ll be adding many more. If you only take away one thing from all of this writing I hope it is the value of tests - we could have saved a huge amount of time by getting those in place earlier.

There was one other last-minute problem with key handling that got confused with the situation I described above, though it is really a completely separate issue. Mac OS X sends key events into Cocoa apps via confusing and inconsistent paths (performKeyEquivalent: vs. keyDown:, either or both in different orders, plus sometimes we don’t get key up events via keyUp:, sometimes we get key up events but they come in via a second call to performKeyEquivalent:, what a mess Apple!). The circumstances that led to issues with this being a problem at the last minute were basically the same as for the other issues I described.

A big thank-you goes out to Robert O’Callahan, Karl Tomlinson, Matthew Gregan, and Masayuki for working so hard to get this situation under control over the past week. Everyone worked together really well, it was some impressive teamwork.