You can also follow a number of our contributors (some of whom don't blog) on Twitter.
BMO 2014 update part II
The second half of 2014 was spent finishing up some performance work and shifting into usability improvements, which will continue into 2015. More performance! By the end of 2014, we’d managed to pick most of the low-to-medium-hanging fruit in the world of Bugzilla server-side performance. The result is approximately doubling the performance of authenticated bug views. Here are graphs from January 2014 and October 2014: The server now also minifies and concatenates JavaScript and CSS files.Searching Bugzilla
BMO currently supports five—count ‘em, five—ways to search for bugs. Whenever you have five different ways to perform a similar function, you can be pretty sure the core problem is not well understood. Search has been rated, for good reason, one of the least compelling features of Bugzilla, so the BMO team want to dig in there and make some serious improvements. At our Portland get-together a couple weeks ago, we talked about putting together a vision for BMO.Making Bugzilla Searches Faster
People often wonder how to make searches in Bugzilla faster on large installations. Two things will give you the most bang for the buck:
- Specify you only want open bugs (if that’s true)
- Specify a product (and, if you know it, a component) to search
Do those two things, and your searches will be much faster.
Coincidentally enough, Bugzilla’s “Simple Search” (BMO version) allows you to specify precisely those two things.
Search Bugzilla with Yahoo!
The Bugzilla team is aware that there are currently 5 different methods of searching Bugzilla (as explained in yesterday’s presentation) – Instant Search, Simple Search, Advanced Search, Google Search and QuickSearch. It has been argued that this is too many, and that we should simplify the options available – perhaps building a search which is all three of Instant, Simple and Quick, instead of just one of them. Some Bugzilla developers have sympathy with that view.
I, however, having caught the mood of the times, feel that Mozilla is all about choice, and there is still not enough choice in Bugzilla search. Therefore, I have decided to add a sixth option for those who want it. As of today, December 1st, by installing this GreaseMonkey script, you can now search Bugzilla with Yahoo! Search. (To do this, obviously, you will need a copy of GreaseMonkey.) It looks like this:
In the future, I may create a Bugzilla extension which allows users to fill the fourth tab on the search page with the search engine of their choice, perhaps leveraging the OpenSearch standard. Then, you will be able to search Bugzilla using the search engine which provides the best experience in your locale.
Viva choice!
Bugzilla for Humans, II
In 2010, johnath did a very popular video introducing people to Bugzilla, called “Bugzilla for Humans“. While age has been kind to johnath, it has been less kind to his video, which now contains several screenshots and bits of dialogue which are out of date. And, being a video featuring a single presenter, it is somewhat difficult to “patch” it.
Enter Popcorn Maker, the Mozilla Foundation’s multimedia presentation creation tool. I have written a script for a replacement presentation, voiced it up, and used Popcorn Maker to put it together. It’s branded as being in the “Understanding Mozilla” series, as a sequel to “Understanding Mozilla: Communications” which I made last year.
So, I present “Understanding Mozilla: Bugzilla“, an 8.5 minute introduction to Bugzilla as we use it here in the Mozilla project:
Because it’s a Popcorn presentation, it can be remixed. So if the instructions ever change, or Bugzilla looks different, new screenshots can be patched in or erroneous sections removed. It’s not trivial to seamlessly patch my voiceover unless you get me to do it, but it’s still much more possible than patching a video. (In fact, the current version contains a voice patch.) It can also be localized – the script is available, and someone could translate it into another language, voice it up, and then remix the presentation and adjust the transitions accordingly.
Props go to the Popcorn team for making such a great tool, and the Developer Tools team for Responsive Design View and the Screenshot button, which makes it trivial to reel off a series of screenshots of a website in a particular custom size/shape format without any need for editing.
Email Filtering on bugzilla.mozilla.org
Vanilla Bugzilla lets you decide which bugmail you receive based on what changed about a bug. But there are a couple of extensions which give you even more control, and both are installed on bugzilla.mozilla.org. So the new email filtering pipeline is as follows:
Firstly, ComponentWatching, as the name implies, lets you “watch” particular products or components, so you get put on the list to receive bugmail for all changes to any bugs in those products or components. This is very useful if you have an interest in a particular area of the project. You can also watch particular users – that function is built-in.
Secondly, the normal email filters run, which exclude or include you from emails based on the particular fields which have been changed in the bug update.
Lastly, the BugmailFilter extension allows you to define “include” or “exclude” rules based on any one of:
- the field changed
- the current product
- the current component
- your relationship to the bug
- who made the change (useful to exclude changes made by bots).
Using these three capabilities in tandem, it should be possible to carefully control how much bugmail you receive, even if you are using a system like Gmail which does not have good client-side filtering.
BMO show_bug Load Times 2x Faster Since January
The load time for viewing bugs on bugzilla.mozilla.org has got 2x faster since January. See this tweet for graphical evidence.
If you are looking for a direction in which to send your bouquets, glob is your man.
New Class of Vulnerability in Perl Web Applications
We did a Bugzilla security release today, to fix some holes responsibly disclosed to us by Check Point Vulnerability Research, to whom we are very grateful. The most serious of them would allow someone to create and control an account for an arbitrary email address they don’t own. If your Bugzilla gives group permissions based on someone’s email domain, as some do, this could be a privilege escalation.
(Update 2014-10-07 05:42 BST: to be clear, this pattern is most commonly used to add “all people in a particular company” to a group, using an email address regexp like .*@mozilla.com$. It is used this way on bugzilla.mozilla.org to allow Mozilla Corporation employees access to e.g. Human Resources bugs. Membership of the Mozilla security group, which has access to unfixed vulnerabilities, is done on an individual basis and could not be obtained using this bug. The same is true of BMO admin privileges.)
These bugs are actually quite interesting, because they seem to represent a new Perl-specific security problem. (At least, as far as I’m aware it’s new, but perhaps we are about to find that everyone knows about it but us. Update 2014-10-08 09:20 BST: everything old is new again; but the level of response, including changes to CGI.pm, suggest that this had mostly faded from collective memory.) This is how it works. I’m using the most serious bug as my example. The somewhat less serious bugs caused by this pattern were XSS holes. (Check Point are going to be presenting on this vulnerability at the 31st Chaos Communications Congress in December in Hamburg, so check their analysis out too.)
Here’s the vulnerable code:
my $otheruser = Bugzilla::User->create({
login_name => $login_name,
realname => $cgi->param('realname'),
cryptpassword => $password});
This code creates a new Bugzilla user in the database when someone signs up. $cgi is an object representing the HTTP request made to the page.
The issue is a combination of two things. Firstly, the $cgi->param() call is context-sensitive – it can return a scalar or an array, depending on the context in which you call it – i.e. the type of the variable you assign the return value to. The ability for functions to do this is a Perl “do what I mean” feature.
Let’s say you called a page as follows, with 3 instances of the same parameter:
index.cgi?foo=bar&foo=baz&foo=quux
If you call param() in an array context (the @ sigil represents a variable which is an array), you get an array of values:
@values = $cgi->param('foo');
-->
['bar', 'baz', 'quux']
If you call it in a scalar context (the $ sigil represents a variable which is a scalar), you get a single value, probably the first one:
$value = $cgi->param('foo');
-->
'bar'
So what context is it being called in, in the code under suspicion? Well, that’s exactly the problem. It turns out that functions called during hash value assignment are evaluated in a list context. However, when the result comes back, that value or those values are assigned to be part of uthe hash as if they were a set of individual, comma-separated scalars. I suspect this behaviour exists because of the close relationship of lists and hashes; it allows you to do stuff like:
my @array = ("foo", 3, "bar", 6);
my %hash = @array;
-->
{ "foo" => 3, "bar" => 6 }
Therefore, when assigning the result of a function call as a hash value, if the return value is a single scalar, all goes as you would expect, but if it’s an array, the second and subsequent values end up being added as key/value pairs in the hash as well. This allows an attacker to override values already in the hash (specified earlier), which may have already been validated, with values controlled by them. In our case, real_name can be any string, so doesn’t need validation, but login_name most definitely does, and it already has been by the time this code is called.
So, in the case of the problematic code above, something like:
index.cgi?realname=JRandomUser&realname=login_name&realname=admin@mozilla.com
would end up overriding the already-validated login_name variable, giving the attacker control of the value used in the call to Bugzilla::User->create(). Oops.
We found 15 instances of this pattern in our code, four of which were exploitable to some degree. If you maintain a Perl web application, you may want to audit it for this pattern. Clearly, CGI.pm param() calls are the first thing to look for, but it’s possible that this pattern could occur with other modules which use the same context-sensitive return feature. The generic fix is to require the function call to be evaluated in scalar context:
my $otheruser = Bugzilla::User->create({
login_name => $login_name,
realname => scalar $cgi->param('realname'),
cryptpassword => $password});
I’d say it might be wise to not ever allow hash values to be assigned directly from functions without a call to scalar.
Bugzilla 1,000,000 Bug Sweepstake Results
Milestone bugzilla.mozilla.org bug 1,000,000 was filed on 2014-04-23 at 01:10 ZST by Archaeopteryx (although rumour has it he used a script, as he also filed the 12 previous bugs in quick succession). The title of the bug was initially “Long word suggestions can move/shift keyboard partially off screen so it overflows” (a Firefox OS Gaia::Keyboard bug, now bug 1000025), but has since been changed to “Celebrate 1000000 bugs, bring your own drinks.”
The winner of the sweepstake to guess the date and time is Gijs Kruitbosch, who guessed 2014-04-25 05:43:21 – which is 2 days, 4 hours, 33 minutes and 5 seconds out. This is a rather wider error, measured in seconds, than the previous sweepstake, but this one had a much longer time horizon – it was instituted 9 months ago. So that’s an error of about 0.95%. The 800,000 bug winner had an error of about 1.55% using the same calculation, so in those terms Gijs’ effort is actually better.
Gijs writes:
I’m Dutch, recently moved to Britain, and I’ll be celebrating my 10th “mozversary” sometime later this year (for those who are counting, bugs had 6 digits and started with “2” when I got going). Back in 2004, I got started by working on ChatZilla, later the Venkman JS debugger and a bit of Firefox, and last year I started working on Firefox as my day job. Outside of Mozilla, I play the piano every now and then, and try to adjust to living in a nation that puts phone booths in its cycle paths.
The two runners-up are Håvard Mork (2d 14h 50m 52s out) and Mark Banner (8d 8h 24m 36s out). Håvard writes:
My name is Håvard Mork. I’m a Java software developer, working with Firefox and web localization to Norwegian. I’ve been involved with localization since 2003. I think localization is rewarding, because it is a process of understanding the mindset of the users, and their perception of IT.
I’m surprised that my estimate came that close. I spent almost an hour trying to figure out how much bug numbers grow, and estimate the exponential components. Unfortunately I lost the equation, so need to start over for the 2,000,000 sweepstakes…
Mark writes:
I’m Mark Banner, also known as Standard8 on irc, I work from home in the UK. I came to Mozilla through volunteering on Thunderbird, and then working at Mozilla Messaging. I still manage Thunderbird releases. Alongside those, I am working on the Loop project (formally Talkilla), which is aiming to provide a real time communications service for Mozilla products, built on top of WebRTC.
Gijs will get a Most Splendid Package, and also a knitted thing from Sheeri as a special bonus prize! The other winners will receive something a little less splendid, but I’m sure it’ll be worth having nevertheless.
bugzilla.mozilla.org Stats At 1,000,000
Thanks to glob, we’ve got some interesting stats from BMO as it crosses the 1M bug mark.
Statuses
UNCONFIRMED 23745 NEW 103655 ASSIGNED 8826 REOPENED 3598 RESOLVED 640326 VERIFIED 220235 CLOSED 1628
Resolutions
RESOLVED
DUPLICATE 119242 EXPIRED 10677 FIXED 303099 INCOMPLETE 30569 INVALID 58096 MOVED 27 WONTFIX 36179 WORKSFORME 82437
VERIFIED
DUPLICATE 64702 EXPIRED 27 FIXED 108935 INCOMPLETE 1746 INVALID 17099 MOVED 150 WONTFIX 6105 WORKSFORME 21471
- Total bugs fixed (RESOLVED/FIXED + VERFIED/FIXED): 412034
- Total duplicates: 183944
Bugs Filed Per Day (April)
2014-04-01 519 2014-04-02 531 2014-04-03 620 2014-04-04 373 2014-04-05 133 2014-04-06 132 2014-04-07 544 2014-04-08 622 2014-04-09 597 2014-04-10 571 2014-04-11 467 2014-04-12 156 2014-04-13 170 2014-04-14 573 2014-04-15 580 2014-04-16 574 2014-04-17 619 2014-04-18 356 2014-04-19 168 2014-04-20 118 2014-04-21 445 2014-04-22 635 2014-04-23 787 2014-04-24 562 2014-04-25 498 2014-04-26 173
Busiest Days Ever
2013-12-30 1360 (bulk import from another tracker) 2013-12-29 1081 (bulk import from another tracker) 2008-07-22 1037 (automated security scanner filing bugs) 2012-10-01 1013 (Gaia bugs import) 2014-02-11 805 2014-04-23 787 2014-02-04 678 2013-01-09 675 2013-11-19 647 2014-04-22 635
User Activity
- We think the earliest bug filed by someone who is still involved with Mozilla is bug 283, which was filed by Wan-Teh Chang on 1998-04-29.
- 2263 people who logged into Bugzilla at some point in April (i.e. are active users) have filed more than 10 bugs.
- The most active user by far is bz:
Bugs filed 4351 Comments made 148493 Assigned to 4029 Commented on 56138 QA-Contact 8 Patches submitted 8080 Patches reviewed 14872 Bugs poked 66215
(You can find these stats about yourself by going to your own user profile. If you are logged in, you can search for other users and see their stats.)
Top 10: Assignees
nobody@mozilla.org 349671 mscott@mozilla.org 16385 bugzilla@blakeross.com 15056 asa@mozilla.org 13350 sspitzer@mozilla.org 11974 bugs@bengoodger.com 10995 justdave@mozilla.com 4768 sean@mozilla.com 4697 oremj@mozilla.com 4672 mozilla@davidbienvenu.org 4273
Top 10: Reporters
jruderman@gmail.com 8037 timeless@bemail.org 6129 krupa.mozbugs@gmail.com 5032 pascalc@gmail.com 4789 bzbarsky@mit.edu 4351 philringnalda@gmail.com 4348 stephen.donner@gmail.com 4038 dbaron@dbaron.org 3680 cbook@mozilla.com 3651 bhearsum@mozilla.com 3528
Top 10: Commenters
tbplbot@gmail.com 347695 bzbarsky@mit.edu 148481 philringnalda@gmail.com 65552 dbaron@dbaron.org 58588 ryanvm@gmail.com 50560 bugzilla@mversen.de 48840 gerv@mozilla.org 48704 roc@ocallahan.org 47453 hskupin@gmail.com 43596 timeless@bemail.org 42885
Top 11: Patches Attached
bzbarsky@mit.edu 8080 dbaron@dbaron.org 4879 ehsan@mozilla.com 4502 roc@ocallahan.org 4397 masayuki@d-toybox.com 4079 neil@httl.net 3930 mozilla@davidbienvenu.org 3890 timeless@bemail.org 3739 brendan@mozilla.org 3659 bugs@pettay.fi 3530 wtc@google.com 3411
Top 11: Reviews
roc@ocallahan.org 15581 bzbarsky@mit.edu 14869 neil@httl.net 9424 jst@mozilla.org 8352 dbaron@dbaron.org 8103 benjamin@smedbergs.us 7272 mozilla@davidbienvenu.org 6198 dveditz@mozilla.com 5983 asa@mozilla.org 5499 mark.finkle@gmail.com 5346 gavin.sharp@gmail.com 5126