Reviewing the Review Process

This is my first planet.mozilla.org post, so you might not know who I am. I’m Justin Scott (fligtar) and I’m an AMO developer, currently working on the developer/admin pages of the Remora project.

Recently, there’s been a lot of feedback through various channels on the AMO reskinning, “spyware” toolbars, and other issues, and I get the feeling that users think their comments are falling upon deaf ears, or not even falling in audible range. This is not the case. AMO 1.0/2.0 is frozen so that our development efforts can be focused on Remora, which you may confuse with us not caring about fixing problems with it. Issues brought up in bug reports, IRC, and other channels are definitely being considered and implemented in Remora.

Last week at the Firefox 2 Summit, we came up with a solution to many of these problems that will raise the quality bar and also increase the transparency of the add-on review process. This new system will divide addons.mozilla.org into two sections: a public area (similar to the current AMO) and a sandbox area. The graphic below is a basic summary of how it will work.

We hope that this process will:

  • Eliminate approval delays for updates to trusted extensions
  • Keep add-ons that are not ready for prime-time out of the general public area
  • Automatically run security checks on all submitted add-on code and flag for security review if triggered
  • Provide a home for Mozilla Labs creations not ready for mass use

While in the sandbox, regular users will be able to install and test the add-ons and then write reviews as to whether the add-on works and should be pushed to the public site. After several of these reviews, an AMO editor will review the add-on and the public reviews and make a decision to send it to the public area or keep it in the sandbox. We expect that some add-ons will stay in the sandbox forever, such as those automatically generated by certain websites.

Please see our meeting notes in the Meeting Notepad or append to the Idea Dump for any ideas you may have regarding the development of Remora.