A few weeks ago we had a very strange problem with a release. We didn’t have the time to investigate it, so we added a hack to workaround it and carried on. I’ve been slowly digging deeper into the issue this week and finally got to the bottom of it. As it turns out the root problem was that our release builds weren’t using the same “make” as the non-release builds! This is absolutely unintended, and could’ve been the source of even more, subtler errors. Eventually, we would’ve hit a problem that we couldn’t have hacked around, and would’ve had to make scary changes under time pressure to fix it.
This is a great example of why it’s important to remove hacks from your code and take the time to understand the errors you hit. You can clean your house all you want but if you don’t find out where the mess is coming from you’ll never keep up with it.
With apologies for the short notice, this post is to let you all know that 32-bit linux tests will be completely disabled on mozilla-central, mozilla-inbound, try, and all other mozilla-central-based branches for the next few days. Our B2G developers are working very hard to finish up 1.0, and need every cycle they can get on these machines to aid them in that.
A few notes, for clarity:
* 32-bit Linux builds are completely unaffected
* “make check” tests will still be run, because they run as part of the build job
* All of the disabled jobs will be re-enabled next week
Again, apologies for the short notice and we appreciate your understanding.
For as long as I can remember #build has been the Release Engineering (née Build & Release) home on IRC. We’ve learned over the years that most people who stop by #build randomly ask questions related to the build _system_, not the infrastructure. This ends up with us redirecting them to #pymake, #developers, or other channels where they’re more likely to get help. In order to help avoid this confusion we’ve decided to move RelEng to #releng and let #build be the home for the build system, likely to replace #pymake.
If you’ve got build infrastructure questions or just like to hang out with us, join us in #releng!