AMO Statistics Dashboard

Yesterday, Mike announced the public preview of the upcoming changes to addons.mozilla.org (AMO). One of the new features that has been long-requested is the ability for developers to see how many update pings, or Active Daily Users, their add-ons have. Just like Firefox, extensions check for updates once a day, and we count how many times this happens for each extension. While the total number of downloads tells add-on authors about how many people have tried out their extension, the active daily user count tells them about how many people used it on a given day, although it’s not perfect.

There’s a bit of fine print regarding active daily usage, but some of the more important points are:

• Only add-ons that do not have an updateURL specified are counted. All add-ons are required to have an empty updateURL when submitted to AMO. If an add-on is distributed from another website with an updateURL, those pings are not counted by us.
• Active Daily Users is not the same as saying “this many people use my extension”. Not all extension users use Firefox every day of the week, users can manually check for updates which will count false active users, etc.
• Many people keep extensions installed but disabled. The stats dashboard allows you to see the various statuses, such as enabled, disabled, incompatible, etc.

Now that some background information is out of the way, on to the features!

Summary Graph and Table

The first thing you’ll see in the stats dashboard is a summary graph that shows the downloads per day and active daily users for the last 2 weeks. Active daily users are currently only counted on Wednesdays, but will switch to every day in the future when we change around the back-end processing. Below the graph is a summary table that shows download information on the left and active daily user information on the right.

Customizable Graphs

The plot selector dropdown in the top left allows you to choose a graph to display: Downloads, Active Daily Users, or Active Daily Users broken down by Add-on Version, Application, Add-on Status, or Operating System. Downloads and Active Daily Users will give a historical graph since July 2007 (or later if your add-on is newer). The ADU breakdown graphs will sort the values by total count and plot the top 6 so that the graph isn’t horribly cluttered. You can use these plot dropdowns to change any of the plots to another value, or even use the options menu at the right to add more plots.

All of these graphs have Firefox and add-on events layered on them so that you can see where the release of a new Firefox or add-on version might have influenced download or ADU numbers. The options menu also allows you to make the graph larger and view the CSV file of the data used in the graph.

RSS

There’s now an RSS feed of the Statistics Summary box updated every day with the latest download and active daily user totals.

Public vs. Private

By changing the settings of their dashboard, add-on authors can designate their statistics as public or private. All dashboards are private by default, meaning only the add-on authors and Mozilla can see the statistics, with the exception of Total Downloads. The number of total downloads was displayed publicly prior to March 2006, and with the upcoming changes will once again be displayed on the add-on’s display page. Active Daily Users and the detailed breakdowns, however, have never been public before and are only part of the dashboard. If the dashboard is set to public, a link will appear on the add-on’s display page to view the statistics, and anyone browsing the statistics index will see a list of all public stats add-ons to view.

I’ve marked one of my add-ons as public, so feel free to take a look at the dashboard yourself. (You don’t even have to log in!)

Known Issues

There are number of known issues filed and a number of known issues in my head for which I haven’t yet filed bugs. Some of the more annoying issues are actually Timeplot (graphing library) bugs that we’ll work on fixing, such as the value popups overlapping and being hard to identify with a plot. I’m also planning on adding a good bit of documentation to the stats dashboard so that you don’t have to read this blog post to understand how everything works.

Feedback

Feedback on the Stats Dashboard can go to the feedback wiki, or for specific bugs and features, please file a bug.

State of the Projects, 2007

Update of 2006 State of the Projects.

Free-time project status:

• Password Exporter - Finished version 1.1 this week and is currently awaiting localization to the 20 languages it’s now in. Should be released sometime the first week of January. More details on that version will be in an upcoming post. Password Exporter has now been download over 203,000 times and has about 70,000 active users every day.
• FavLoc - Haven’t had time to work on it. Will try to find time to update compatibility for Firefox 3 sometime soon, but doubt I’ll add any new features. 28,000 downloads and 6,000 active users.
• LSU Campus Map - I haven’t updated anything since I first made it a few years ago, but it’s still very popular with people just searching for an LSU campus map. It had especially high traffic following the recent shooting on campus.
• All other projects including arraise, Startup Authenticator, Degree Analyzer, elurt, and all other ideas in my head that I detail but never start - I haven’t had time to work on and are pretty much dead.

Work project status:

• addons.mozilla.org - Remora was launched in March and the AMO team is currently working on version 3.2 to be launched in mid-January. I’ve been working on a statistics dashboard for add-on developers, and after 3.2 will be continuing to work mainly on AMO stuff, probably including a Developer Control Panel revamp with lots of new features.
• Operation Firefox - Contest was a big success and winners were announced earlier this month. Site will probably stay the way it is.
• Extend Firefox - Contest ends tonight. Future plans to be announced later.
• Rock Your Firefox - 0.6 released. Work on other milestones is not currently scheduled, and the app is pretty much in maintenance mode right now.
• Personas - Site hasn’t officially launched yet, but has been finished for a few weeks.
• Misc. other projects popping up and going away just as quickly.

The last few days I’ve been cleaning up a lot of site stuff, as I was quite shocked to find I had over 50 subdomains on this site, many of which are no longer used. I think I’ll be removing the ribbon from my pages soon, as I don’t think it’s helping anything.

My 2008 Add-on Developer Tools Wishlist

Statistics Dashboard

A statistics dashboard for add-on developers that shows historical graphs of downloads and active users based on update pings. The active user data should include breakdowns for the add-on version in use, the application and version, operating system, and add-on status (enabled, disabled, incompatible, etc). This is already being implemented and, if all goes according to plan, will launch (at least on a preview site) by the end of 2007 populated with data from the last 6 months.

Developer Control Panel Revamp

AMO has a lot of new ideas and a lot of new users. The Dev CP is better than it was pre-Remora, but can still come a long way in user experience and discoverability. The public side of AMO will be getting a makeover soon, and following that I’d like to redo the Developer side to support many of the new principles we’re trying to implement.

Localization of Extensions

Although the AMO site itself is available in 20 languages and more every few months, many of the extensions hosted on the site are only available in one. The site is translated, but does not have a truly localized feel to users in other locales. Wil Clouser has been working with the L10n drivers and I think AMO will have a plan soon for how we’ll facilitate the localization of more extensions so that when someone visits the site from Japan, they see featured add-ons that are popular in Japan and have Japanese descriptions and language packs.

Taggable Extension Code Repository

The Mozilla Security group has long wanted a searchable repository of extension code to look for vulnerabilities and also to know if extensions will break if a certain part of Firefox is changed. These searches are currently done manually by AMO admins. A taggable extension code repository would support that, as well as provide extension developers with a large pool of example source code.

When uploading an add-on to AMO, developers would be asked if they’d like their extension to be listed in the extension code repository. The repository would have a wiki-style tagging system, in which anyone could select specific lines of an extension’s file(s) and tag them as “opening a new tab” or “creating a new about: page”. (I volunteer Mark Finkle to be the wiki gardener!)

A simpler version of this that schrep suggested is checking extension source into a CVS repository and using existing tools like LXR/MXR.

Add-on skeletons

A few weeks ago I noticed that Facebook has made it ridiculously easy to create an application. There’s now a “Check out example code for this application” link under each of your applications that pops up a dialog with 3 easy steps to create a working application, two of which are copy and paste and the third is clicking a link.

I was trying to think of ways that we could do the same for add-ons. I know there are already various tools available for creating basic structure for install.rdf and such. Chris Beard suggested creating skeleton extensions for each of the major types of extensions, such as a statusbar extension, sidebar extension, toolbar, theme, etc. This would make it very easy for new developers to dive right into working code and not spend time trying to figure out the directory structure of an extension.

Along with the basic skeletons, we may want to provide “tutorial” skeletons that are extensively documented. For example, where we include a JavaScript file in XUL, we have a comment that explains why you should never include a remote JavaScript file in an extension.

I don’t expect all of these things to happen in 2008, but it’d be cool to see a few of them.

Stupid Filter

“Because the internet needs prophylactics for memetically transmitted diseases.”

There’s a new open source project at StupidFilter.org to create filtering software that can detect “stupid” comments on the internet, such as comments on YouTube and similar sites. While this is an interesting project, the part that really caught me is that they’re going to release it as a Firefox extension. (They actually said plugin, but I’m guessing they meant extension.) While they mention other forms of delivery, all but the extension are aimed at the hosts of the content, not the consumer.

When I first read about the project on Slashdot, I thought “I hope they make it a Firefox extension”. It’s exciting that so many of the ideas and innovations people have these days are implemented through Firefox add-ons. Hopefully we’ll see even more of that in the future, as Mozilla is investing heavily in its add-ons community and I expect the results of that will be apparent in Firefox 3 and the upcoming changes to the add-ons website, as well as some much awaited new tools for add-on developers that I’ll be blogging about in the coming weeks on the WebDev blog.

Top Ten Add-ons

I’ve wanted to make this post for awhile and never got around to it, but a recent conversation reminded me about it. Below are the “top 10″ add-ons (extensions and themes) collected three different ways. (This table best viewed on my actual blog.)

Total Downloads and Weekly Downloads are the methods currently used for the “Most Popular” searches on AMO, and represents the number of downloads of that add-on from addons.mozilla.org.

Active Users is based on the number of daily update pings, and is a better metric of the popularity of an add-on because it shows which are actually being used after downloading. This information is not yet available on AMO, but we’re working on it.

Facebook Favorites is the number of users that have added that add-on to their favorites/recommended list in Rock Your Firefox. This shows add-ons that are both in use and loved. It would be a better metric of both popularity and quality than the others if the data wasn’t skewed a bit in that people that use Rock Your Firefox tend to be more technically inclined. Once we bring the favorites system to AMO, this data will be more reliable.

Upcoming Facebook Platform Changes

Tonight, Facebook announced some upcoming changes to their platform. I wanted to comment about how they will affect Rock Your Firefox.

• Profile box visibility changes - This affected our default text shown before a user adds any favorites, but has already been fixed. Developers were previously told they had 10 days to adjust this, however that deadline has now been extended.
• Application Directory metrics shift to user engagement - I imagine this could only be better for us, as our statistics show that most of our users check back with the application every day. [Aside: We have fought with a similar problem on AMO of gaging popularity of add-ons based on raw download counts: People try to game the system to get into the top 5, which then gets them in for a long time as people will download anything in the top 5 without much consideration and then uninstall it. We will have new statistics to gage this by soon, including active user counts and "favorite" counts from both Facebook and AMO, which provide a much better idea of which add-ons people actually use and like.]
• Requests/Invitations deprecation - Rock Your Firefox does not allow users to send invitations to other users. A large part of that is because I expected this to happen. Application invitations are too spammy, and I didn’t think they would last long, especially with some applications abusing them by sending when the user didn’t even select friends to invite. Facebook stated that a standard tool for this will be offered, which is exciting, as we get the functionality but not the wasted time on development of the soon-to-be-deprecated system.
• No more e-mail notifications - Does not affect Rock Your Firefox.
• Newsfeed stories for everyone - As an application developer, this is very exciting. Currently, newsfeed stories generated by applications are only seen by friends that also have the application added. The only newsfeed story that has a chance of attracting new users is the generic “Foxkeh has added the Rock Your Firefox application.” If all stories have a chance of being seen, we can generate a lot more interest from users that don’t know about add-ons or don’t even know about Firefox. As a Facebook user, I really hope they have good controls so that if I don’t want to constantly see someone superfeeding their pet gift that grows into a map of places you’ve been, I don’t have to.
• Join Facebook with an application already added - Earlier this month, Facebook posted that they are working on a feature to allow users to sign up and immediately start using an application. As we have been promoting the rockyourfirefox.com URL and I suspect that several community members joined Facebook just to see the application, this will be helpful.

Overall, these changes are great for us, but I’m sure they are quite unfortunate for other applications that use functionality that will now be removed. Rock Your Firefox 0.4 is scheduled to release this Friday with two awesome new features, which I’ll blog about once they’re live.

A Comparison

I was reading a post about it being Facebook Platform’s 3 month anniversary and decided to do a little comparison.

* Only considers add-ons hosted on addons.mozilla.org; there are many add-ons hosted by their individual developers not taken into account.

I’m not one to get into the politics of whether Facebook Platform is good for the web. I’m more interested in what we can do to have people as excited about Firefox add-ons as they are about getting that one person who always takes forever in Scrabulous to finally go.

What can we learn from Facebook Platform? Why do people so readily add applications?

• Applications are restricted. People know exactly what an application is allowed to do within Facebook because you see a list of checkboxes when you add it. It can’t cause data loss. It can’t modify your information. Extensions can do anything they want. They can cause data loss. They can do bad things to your computer. I think that Firefox users who have never tried extensions are either: a) confused about what extensions are, or b) concerned about their security
• Applications are required to view content. If someone sends you something in an application, you often have to add that application to see it. In Firefox, we call those plug-ins. I can’t think of any extensions that you have to add in order to see content you find scattered around the web.
• You know when friends add applications. People add applications when they see something cool that a friend added. With Firefox add-ons, you don’t know what add-ons your friends use unless you use their browser or they talk about their extensions. (Although many of my friends talk about what extensions they use, I think it is fair to say this is not the norm.) This is what Rock Your Firefox tries to solve: seeing what add-ons your friends use and improving discoverability.

Having said that, I’m not saying that we should make any changes to be more like Facebook. I certainly think that extensions’ ability to modify anything is key to the innovation that we see all the time in new extensions. I think we need to find out what facets of the Firefox ecosystem are the most compelling and make people realize that everything that’s being done on Facebook could be done even better in the open web.

(Obligatory disclaimer: The above opinions are my own and are not affiliated with any entity in which I may be involved.)

Hai (it’s Japanese, not lolcat)

The last few days I’ve been in Tokyo, Japan to take part in the Firefox Developer Conference today, along with Mike Shaver, Mark Finkle, and Dan Mills from the US and Canada. I gave a presentation on Mozilla Add-ons and the add-on submission process and then answered questions about the Future of AMO with the most appreciated assistance of shaver. If you’re interested, my presentation is available here.

Photos and video from the trip so far are up. I’ll be heading home tomorrow evening.

I’d like to give a huge thanks to the folks from Mozilla Japan who did an amazing job of organizing the event today with over 160 people attending and still managed to help us get around the city the last few days. I had a wonderful time and learned a lot about the Firefox and add-on community in this part of the world.

I would like to take this opportunity to say that there was someone that looks exactly like me roaming around today and that any pictures you see are probably of him, not me.

Getting text without gettext

Last week I made the switch from Windows to a Mac OS X. I’m definitely happy with the change, although one aspect gave me quite a bit of trouble: PHP gettext support. Now that AMO is localized (in 20 languages and growing!), gettext is required for a local installation.

To make a long story (involving MAMP, XAMPP, compiling PHP and Apache multiple times, and then back to MAMP again) short: MAMP should really include PHP compiled with gettext, as many other frustrated people have pointed out. During my battle, I asked others on the team that have been using Macs what they do for development. Some do all of their development in vim on a development server and others use a Linux virtual machine.

I had already written a basic .po string parser for the Localizer Control Panel, so I decided to make my own gettext emulator. The result is faketext, and MAMP + faketext = working Remora installation. It quickly parses .po files and will load a page in a specified language just as if you had the PHP extension installed.

Apparently there’s an inactive project called php-gettext that has the same idea but parses the binary .mo files instead. I didn’t try it because it would require many of the same modifications I made to work with AMO, and wenzel says it’s extremely slow anyway.

AMO faketext - save as config-local.php and drop in site/app/config.
Generic faketext - will probably require slight modification to detect the requested locale, as indicated in the comment at the top.

The Cached Edge - addons.mozilla.org (#2)

The following addons.mozilla.org bugs for version 3.0 (Remora) were RESOLVED FIXED between Monday, Feb. 26 and Sunday, March 4.

• Bug 372302 - Plus sign not allowed in URLs - prevents downloads
• Bug 370604 - version range missing in remora
• Bug 367574 - Ability to filter search of an Add-ons by application (Firefox or Thunderbird)
• Bug 370614 - New download URLs make saving extensions difficult
• Bug 370602 - Popular add-ons are sorted by total downloads
• Bug 371231 - Migration script doesn’t account for entities in taglines
• Bug 371940 - Wrong date format for the Russian locale
• Bug 371618 - correct remora for SeaMonkey (section title, logo)
• Bug 372011 - 404s on the discussions pages
• Bug 370600 - Editing version info pushes add-on to top of “New and Updated” list
• Bug 372297 - Bogus sandbox link on add-on info page
• Bug 372010 - Discussions breadcrumbs are incorrect
• Bug 371614 - Several JavaScript strict warnings
• Bug 371780 - The “Build Your Own” link in the navigation menu is hardcoded to english
• Bug 371773 - User is not notified that reviews need editor approval before they appear
• Bug 371173 - Favicon is sent out with text/plain content type
• Bug 371616 - Several stylesheet anomalies
• Bug 372116 - User info page shows disabled add-ons
• Bug 371782 - Previous Versions page shows current version
• Bug 371472 - User Info Page misses full author name if present
• Bug 370774 - Language selection dropdown is smaller than lawyer smallprint