Lifecycle of a Release Engineering bug

Like most other engineering groups at Mozilla, Release Engineering uses Bugzilla to track most of the work we do. Our bugs and patches use many of the same flags as a Firefox bug, so it's no surprise that many people expect our bugs to be like Firefox bugs in other ways. I've noticed a few instances of confusion recently, so I'd like to try to clear things up a little bit by talking about what's different about ours.

In order to understand our bugs, a brief overview of our repositories is required. Unlike Firefox, RelEng hosts its code in a number of different Mercurial repositories. Most of these repositories have (or should have) a "production" branch in addition to the "default" one. Our production systems (Buildbot masters/slaves, AWS management machines, signing servers, etc.) track the production branches of our repositories. This is important for a few reasons:

• Some patches can depend on other patches in a different repository
• Some patches need to be deployed at specific times
• Some systems cannot automatically use new code
• Every commit has the potential to close the trees

Once a patch has been reviewed it will be landed on the default branch of its repository. At this time the checked-in attachment flag is set to "+". Unlike Firefox bugs, the bug stays open at this point. Many of our repositories have their own continuous integration tests that watch the default branches and do as much up-front verification as they can. Like mozilla-central, we do our best to stay in a shippable state at all times - so if we have test failures, they get fixed quickly or backed out. Sometime later (usually about once per day), someone will merge all of the pending patches to the production branches and make sure the production systems pick them up. Once this is done, they will update the Maintenance page and add a note like "In production" to all of the bugs that had patches merged. Any new jobs that start after this point will be using all of the new code. In most cases bugs will be left open, and closed by their assignee when appropriate.

I hope this helps the next time you're confused about the state of some RelEng work!