Over the years we've had quite a few run-ins with Windows' maximum path and command line lengths. These problems exhibit themselves with a cryptic "Bad file number" error and usually leave someone scratching their head for a bit. Due to the variable length of our build directories these problems are prone to biting us at inconvenient times - in particular, during release builds. They also have the potential to bite us on merge day or anytime cross-branch merging happens.
To prevent this from happening in the future I'm going to be landing a change to our build directory naming this week. After it lands we'll be padding build directories with "0" until they reach the required length. For example, "try-w32" becomes "try-w32-0000000000000000000000" and "rel-m-beta-w32_bld" becomes "rel-m-beta-w32_bld-00000000000". Because all build directories across all branches (including try) will be consistent, this means that any changes that would've eventually failed because of path or command line length, will now fail during the original landing.
This change should be mostly invisible to developers, except for the extra characters in paths when looking at build logs.
A few weeks ago we had a very strange problem with a release. We didn't have the time to investigate it, so we added a hack to workaround it and carried on. I've been slowly digging deeper into the issue this week and finally got to the bottom of it. As it turns out the root problem was that our release builds weren't using the same "make" as the non-release builds! This is absolutely unintended, and could've been the source of even more, subtler errors. Eventually, we would've hit a problem that we couldn't have hacked around, and would've had to make scary changes under time pressure to fix it.
This is a great example of why it's important to remove hacks from your code and take the time to understand the errors you hit. You can clean your house all you want but if you don't find out where the mess is coming from you'll never keep up with it.
Hi folks,
With apologies for the short notice, this post is to let you all know that 32-bit linux tests will be completely disabled on mozilla-central, mozilla-inbound, try, and all other mozilla-central-based branches for the next few days. Our B2G developers are working very hard to finish up 1.0, and need every cycle they can get on these machines to aid them in that.
A few notes, for clarity:
* 32-bit Linux builds are completely unaffected
* "make check" tests will still be run, because they run as part of the build job
* All of the disabled jobs will be re-enabled next week
Again, apologies for the short notice and we appreciate your understanding.
- Ben
For as long as I can remember #build has been the Release Engineering (née Build & Release) home on IRC. We've learned over the years that most people who stop by #build randomly ask questions related to the build _system_, not the infrastructure. This ends up with us redirecting them to #pymake, #developers, or other channels where they're more likely to get help. In order to help avoid this confusion we've decided to move RelEng to #releng and let #build be the home for the build system, likely to replace #pymake.
If you've got build infrastructure questions or just like to hang out with us, join us in #releng!
As I mentioned last week, we've been working hard to get multilocale B2G builds going. This morning we flipped the switch and turned on multilocale Gaia across the board. The existing desktop and device builds on TBPL will now include a Gaia profile with Arabic, English, Spanish, French, Brazilian Portuguese, and Mandarin Chinese (zh-TW).
Additionally, we now have new desktop builds with all of the locales listed in this file available -- meaning that most people localizing B2G can now test their translations in-app. These are available for Linux, Mac, and Windows in the "localizer" packages in this directory.
A few notes:
Linux users who have run a B2G desktop build in the past may need to rm -rf ~/.mozilla/b2g before these new builds will function correctly.
Gecko is still en-US only. Expect network errors and other such things to be in en-US for now. Multilocale gecko for B2G work is being tracked in bug 817197 and the bugs that it blocks.
All of the desktop builds will be updated on a nightly basis. However, there are no automatic updates for them - you must download by hand whenever you want to test newer code or translations.
Currently, all of the b2g builds on TBPL are using a fixed set of locales which are built into the Gaia repository. These were OK at first, but we're at the point now where we need to be able to test the languages that we'll be initially shipping, as well as provide some way for localizers to test out their work. For these reasons, the following changes will be made to the B2g desktop and device builds:
* Unagi, otoro, panda, and the current desktop b2g builds will include 6 locales (instead of the 4 they currently do): Arabic (ar), English (en-US), French (fr), Spanish (es), Brazilian Portuguese (pt-BR), and Mandarin Chinese (zh-TW). These builds are intended for developer consumption and help to test out a wide array of features (rtl languages, languages with long strings, unicode characters, etc).
* We will be adding new desktop b2g builds that contain all languages that Gaia is available in (see https://github.com/mozilla-b2g/gaia/blob/master/shared/resources/languages-all.json for the full list). These builds are intended for localizers to see how their translations look and feel.
These changes should be taking place sometime this week. Big thanks to Staś Małolepszy for adding support for this to the Gaia build system.
Note that for now, the Gecko portions (for example, network error pages in the Browser) will not be localized. We will be enabling localization for Gecko as soon as we can, but it's not quite ready yet.
It's not uncommon to see unit test failures, performance issues, or build problems that seem to happen exclusively on our pool of build & test machines. If you've ever been plagued by one of these you know what a pain in the butt it is to debug such a thing. Because of this, RelEng is always willing to loan out slaves to anyone who needs to do on-machine debugging. All you need to do is file a bug! These requests can usually be turned around in the same business day. If you think having access will save you time, do not hesitate - file today!
I'm an avid fisherman, a beginner gardener and in general someone who loves to do things for himself. Hunting is something I've wanted to try for a very long time (minus the 5 years that I was a vegetarian), and this year I finally got around to getting the proper licenses and doing it.
Preparation
Here in Ontario it's not a trivial process to be licensed to hunt with a gun. One must first pass the RCMP's course for unrestricted firearms, then pass the Ontario Hunter Education Program, and _then_ apply to multiple government agencies for licenses. The background check for getting a gun license is at least as extensive (and probably more) than for getting a passport. The hunting license is pretty trivial once you have that, but there's an array of licensing options to sort through. I took my training courses back in May only finished getting all of my licenses in mid-October -- two weeks before deer season opened. (My experience was a bit non typical because I needed an extra medical clearance, I expect it's a bit quicker for folks who don't.)
After getting my licenses it was time to gear up. There's a lot you need to be properly equipped to hunt. In the end I purchased:
Sweat whisking pants/shirt
Long, cotton underwear+top
Two pairs of thick socks
Blaze orange hat/vest
Two-layer camo jacket
Camo gloves
Thin undergloves, for extra warmth
Multitool
Hunting knife
Compass
Small maglite
...and still had to borrow waterproof camo pants, a rifle and misc. things (ground blind, chair, gun stand, etc.) in order to be fully equipped. I decided to buy a gun for myself this year to get some more hands on time with my friend's gun to figure out my preferences a bit. Excluding the course/license fees I spent about $250 on gear.
I set aside 3 days for hunting this year, and one for getting comfortable with the gun I'd be using. I rolled into town Sunday afternoon and spent some time with my friend's Remington 700, 7mm rifle. It had already been sighted-in so all I needed to do was get comfortable with it. The first thing that was surprising to me was how heavy it was -- a 7mm is NOT a small gun. I don't know the exact weight with the scope but even in a seated position with a gunstand it was impossible for me to hold it still. With that said, I was surprised at how easy it was to shoot and at my accuracy (at a paltry 50 yards). With a 3 shot clip I had about a 3" spread -- for a pro that would be terrible but given that it was my first time shooting a high powered rifle I was quite pleased. I later learned that a 7mm is generally considered hugely overpowered for deer hunting, so next year I'll likely buy something more reasonable and lighter. I shot off a couple of clips without issue, but on the last shot I managed to get whacked by the scope from the recoil. I had a been warned this would happen to be at least once, and then I would get laughed at -- both of those were true.
This picture doesn't do the wound justice -- it bled for 15min and stayed red for days.
The Hunt
The person that I was hunting with is good friends with some folks who own a good portion of land in a rural area of eastern Ontario. They were kind enough not only to let us hunt on their land, but to feed us a huge breakfast every day.
In Ontario, you may hunt between 30min prior to sunrise until 30min after sunset. Factoring in time to shower, get ready, drive over, walk out & get comfortable this meant that we were getting up at 4am and leaving the house at 5. Each day we were sitting down in the blind before 6am and ready to load our guns as soon as it was legal to (you can't load your gun until 30min before sunrise, either). It was below zero every day but other than the tips of my toes and fingers I stayed quite warm by wearing about 5 layers of clothing.
All three days of hunting went pretty much the same. We walked in prior to hunting time, made calls until 9am or so, and then headed in for breakfast. We'd come back out around 10/10:30 and spend the afternoon watching the birds, napping on and off, and just relaxing. (Let me tell you, if you've never had a nap while bundled up at -3C in the middle of the woods, you're missing out. It is freaking amazing.) Around 3pm we'd start focusing again, as dusk is when the deer will start to become more active again. We'd sit as quietly as possible, make calls, and do whatever else we can to bring the deer in. On the final day we found some fresh tracks in the afternoon and were very hopeful that it would return that night. Unfortunately, each day ended without even seeing a deer.
Going in I expected to be very disappointed if I came out empty handed, but that was not the case at all. Even though I regularly fish, have a yearly garden, and occasionally hike I haven't felt this close to nature since I was a cub scout. Spending nearly 12h just sitting, observing, and thinking about nature gives me a new level of appreciation for it. Whether you hunt or not, I can't recommend the experience of observing nature enough.
My view for three days
We're in the process of developing a new version of our update server. It's at the point now where we have a development environment set-up for it, and slaves attempt to submit data to it. Recently we came across an issue where slaves were unable to successfully validate the SSL certificate of the development environment. Specifically, it was raising this error from OpenSSL:
Our client uses the Python Requests library, and gives it a specific certificate to validate the server certificate with. For comparison we tried using openssl's s_client with the same cert, which was able to validate the certificate just fine given the same inputs. HRM.
I tried tons of things to get it to work - different versions of the Requests library, giving it the root + intermediary cert, different versions of Python - but nothing worked! I couldn't even reproduce the behaviour with pure openssl no matter what inputs I gave it. I reached out to #security for support and Yvan couldn't even reproduce the problem on his Mac!
Eventually I reached out to #python-requests, thinking it was a bug in the library. Someone there suggested strace'ing both my python script and openssl. I did that, and found something very interesting: despite being given an explicit certificate bundle, openssl fell back onto the system certificates -- my python script didn't. To verify this, I changed my script to look at the full system root rather than just a specific certificate or bundle. After doing that, everything worked fine.
My belief is that one of PyOpenSSL, urllib3, or Requests doesn't know how to look at the system certificate store at all on Windows or Linux. On Mac, it seems to fall back just fine on it. One thing is certain: security is hard, especially debugging it.
Big thanks to Jake Maul, Rail, Yvan Boily, Dveditz, and #python-requests for helping me debug this.
I've been growing jalapeño peppers this year. My first haul of them came out extremely bland - more like green peppers with almost no spice to them. This happened the last time I tried growing them too, and I nearly gave up hope. After a whole lot of googling I came across multiple suggestions that overwatering can cause bland peppers. I stopped watering my plant immediately and a week later I had spicy jalapeños! In the end I managed to harvest about 25 peppers - more than I ever thought I could get in one season from one plant. With so many peppers to deal with I really had no choice but to make a sauce!
Like my habanero sauce recipe, this jalapeño sauce is largely based on another recipe and rather simple. I didn't have all the listed ingredients on hand so I had to make some modifications:
Ingredients
10 jalapeño peppers (mine were 1/2 spicy and 1/2 bland), finely chopped
3 hot house tomatoes, chopped
1 large onion, chopped
4 cloves garlic, minced
3/4 cup cider vinegar
2 tbsp oregano
salt to taste
Directions
Cook tomatoes, onion, garlic, vinegar, and oregano in a saucepan until softened.
Add the jalapeños and cook until they have softened and the mixture has taken on some of the green colour of the peppers.
Using a blender or food processor, puree until smooth. Depending how much moisture was lost during cooking you may need to add a bit of water.
Pour the mixture back into the saucepan over low heat.
Season with salt to taste.
I brought some of this to the office today and folks found it be quite good as a dip for pita chips. I suspect it would work well as a garnish on sandwiches, burritos, etc. as well.
