Virtual Machines and Real Humans
Encouraging contributors to open source website development
Mozilla, like any good open source project, is a community. 25% of contributions to Mozilla Core, Firefox, Thunderbird, and other products come from the community rather than employees. We, i.e. Mozilla Webdev, want more community web developers to contribute to Mozilla websites. Here are community contribution stats for some Mozilla websites:
- SUMO
github: kitsune
ADV*: 357k
community contributors: 0† - AMO
github: zamboni
ADV*: 736k
community contributors: 3-4† - Socorro
github: socorro
ADV*: ?
community contributors: 1-2† - MDN
github: kuma
ADV*: 56k
community contributors: 4
* Average Daily Visitors
† Guesstimated figures - looked at commit logs and our LDAP phonebook; open to corrections
tofumatt wrote a good post about how we're starting to use vagrant and puppet to give developers easier access to working on Mozilla Webdev sites. lmorchard has a passion for this idea, and he's been extremely patient with all my troubles and questions about Vagrant and Puppet as I've tried to jump on the bandwagon of putting clouds in boxes. I don't want to discredit their work and opinions, but let me offer my own.
I am slow and make mistakes
Instructions for humans are nowhere near as good as instructions for computers. Humans make mistakes, misunderstand, and are generally not as fast as computers when doing this kind of thing.
- tofumatt
Amen to that. If you read some choice excerpts from our #mdn chat log, you'll see how slow I am (and how patient lmorchard is) re: vagrant. Even though I'm slow, I presumably have more "sysadmin" experience than the oft-cited hypothetical design contributor just trying to make a CSS fix, and yet vagrant and puppet make me cringe. So far, vagrant and puppet have only introduced MORE setup pain - and not the regular MySQL setup pain that I (and yes, even designers) have done dozens of times already.
Making boxes put boxes into clouds for your boxes
I've also started playing with Puppet by itself, to build plain-vanilla VM appliances and to spin up cloud servers on Amazon EC2 and Rackspace Cloud Servers. It's going pretty well, and it's conceivable we could do without Vagrant to script the build of downloadable VM appliances and to spin up disposable cloud servers.
- lmorchard
lmorchard is spot on - maybe because he's tired of dragging my slow brain thru vagrant and puppet. Our CI machine should generate vm images with 1-button deploy to Rackspace or Amazon. I think this idea of using VM's to encourage community contribution to website projects is an all-or-nothing deal. If it doesn't work 100%, it's worse than failure. If a community contributor runs into problems with vagrant or puppet, it's much worse than if they run into problems with MySQL or pip. At least there's a bunch of established resources to debug MySQL - I couldn't find much debugging resources for vagrant, besides lmorchard.
At SourceForge we had this kind of "ideal" setup for years. The Service Operations Group at SourceForge maintains sandbox servers with an admin dashboard for creating, managing, and destroying per-developer sandbox vm's that run instances of SourceForge.net. (Though sadly it's behind corporate LDAP - only given to employees) I don't know if/how that model could scale from a single site to dozens of sites all with different moving parts. And there are big latency issues developing on a remote vm. Still, it's exactly what we would want for that elusive CSS designer who wants to fix Mozilla sites without setting one up. She clicks "Create MDN Sandbox" and gets an email containing the domain and access credentials to start editing code on her sandbox. Beautiful.
The virtual machine is us
The web is no longer just linking information ... the web is linking people. People sharing, trading, and collaborating.
In all this technology, we sometimes lose track of the people. I don't want to abandon the idea of using virtual machines to encourage community contribution; the SourceForge sandbox model is, for me, the ideal system for enabling easy contributions from casual community developers. I'll join work for something similar at Mozilla.
I do want to highlight an additional on-ramp for getting community developers involved with Mozilla websites - Mozilla webdev's should get involved with our developer communities!
A couple weeks ago at our Tulsa Web Devs coworking day, Buddy and I were talking about Mozilla and he asked how he can get a job at Mozilla. I said Mozilla really likes looking at your open-source code, and also likes hiring community contributors. He asked how he should become a contributor. The easy answer is go to the Get Involved with Mozilla page, but since I was working on MDN that moment, I showed him the kuma repository I was working on. He forked it and went thru the "menial and often fragile setup" in our installation.rst; I fielded his questions as he went.
It was incredibly valuable - he filed a kuma setup and installation bug and made the appropriate fixes. Then he went on to fix an easy 1-liner bug, and got comfortable enough to do a significant bug involving cron, urllib, and json to generate a humans.txt file for MDN - which now includes him as a community contributor!
Of course, we can't spend all our time recruiting and mentoring community developers to do our work for us. [Or can we? ;) ] And Buddy is an exception in his level of commitment to contribute. So, yes we should reach out to those developers looking to make a casual CSS fix without wanting to set up a whole website environment (though we've received those 1-line CSS commits into MDN even without vm's). That's a good reason to pursue turn-key VM's.
But if you're at a conference or an event and someone asks you how to get involved with or fix a bug on Mozilla websites, don't just send them to a page or a virtual machine - pull up a chair and work with the real human sitting next to you.
['Tulsa %s' % e for e in ('Tech Fest', 'Hackathon')]
Tulsa Tech Fest
In addition to our work on MDN, this is a busy week for me in Tulsa - the annual Tulsa Tech Fest is Friday Oct 7th. Our Tulsa Web Devs group is doing an HTML5 track; my talk is the intro - HTML5: Code for all the platforms! (Soon to be a link to the slides)
TTF is a Microsoft-heavy event, but we've come a long way since TTF 2007 when Steven and I did the entire PHP track ourselves, covering Eclipse & PDT, PHPUnit & Selenium, CI w/ Phing & CruiseControl, DB refactoring w/ dbdeploy - sophisticated topics for 2007!
As far as the Tech Fest goes, I'm really looking forward to the community town hall the night before - developer group leaders from all around Tulsa get together to talk about how to grow the Tulsa developer community. I'm eager to see how the other groups are doing and to share experiences.
I'll spend all of Friday in the HTML5 room, unless I'm pulled off to the
Tulsa Hackathon
Tulsa Hackathon is a big event for me, as I've done a bunch of work for it.
It all started when Tulsa Web Devs took it upon ourselves to put Metro Tulsa Transit Authority data onto Google Maps. After a few hack days we made it work - barely. We still don't have all of MTTA's data, and we didn't make a shape file to smooth out the way Google draws the bus routes to match the roads, but we did pass the GTFS validation test! All that's left is to get MTTA to use our tool to publish the feed for Google - something that's proving harder than any of the code, I think.
In any case, most of us enjoyed working on a cool project that improved Tulsa. So, I reached out to other organizations to see what other projects we might try. Over the last couple months, we've set up projects for INCOG (another transit org in the area), Tulsa City County Library, Community Food Bank of Eastern Oklahoma, and Fab Lab Tulsa. We're an official Random Hacks of Kindness community event!
Scott Phillips and I have arranged all the facilities and venues, logistics, swag, meals, etc. I'm anxious to see how much we actually get done. But really, there are just two goals - improve Tulsa, and our developer community.
Thank you Stormy (and Mozilla) for helping to sponsor our Hackathon! Everyone in Tulsa loves having Mozilla involved! I love that Mozilla promotes the web for public benefit - from hundreds of millions of Firefox users to a few dozen hackers in Tulsa Oklahoma.
The Long Local Tail of Web Developers
At my last work week in Mountain View, I sat in on a few #devengage meetings. I heard someone (maybe Christian), say we do a good job talking to the top 20% of web developers but not necessarily the bottom 80% - the long tail of web developers. At Mozilla, I know I'm not in the "top" 20% of Webdev - there are just too many awesome webdevs around me. And though it's intimidating, it's also exhilarating to work alongside so many top-notch developers who can teach me the "top" 20% of web development skills. (Les, I'm looking at YOU!)
Still, something many of us remote Mozilla employees appreciate is the "rock star" status that working for Mozilla grants us in our local places outside of "Hugeness bubbles" like Silicon Valley. The Mozilla reputation is relatively higher and commands more attention outside these bubbles.
For example, at the Tulsa School of Dev conference, with (I think) 150 attendees, a few Tulsa Web Devs arranged a "web" track that really amounted to a "WordPress" track, since all of us had at least some WordPress experience to share on an entry level. That afternoon, I gave a Web App Security talk using our own WordPress site as the victim. (Sorry guys). My session had about 40-50 attendees - maybe 1/3 of the entire conference!
I'm smart enough to know that I'm NOT smart enough to attract that many developers on my own merit. The simple fact is putting "Mozilla" next to a speaker's name in Tulsa, Oklahoma attracts relatively more developers than it does at High-Profile-Town Tech Conference 2.0. In Portland, at Open Source Bridge with 480 attendees, where more knowledgeable and more adept Mozilla Webdevs spoke; they attracted roughly the same 40-50 number of attendees at each of their talks - including those of us in Webdev!
What if Mozilla leveraged our remote employees' clout to reach out specifically to their local web developer communities - the "bottom" 80% of web developers like me? What would that do? I would hope a few very cool and important things:
- Push & pull more web developers towards the "top" 20% which ...
- Flattens the head of web developer knowledge and skill, which ...
- Raises the overall level of web developers
I've read the Long Tail. I agree power laws are natural and healthy distributions. We need the top 1% and the top 0.1% who are pushing development limits at the razor's edge. At the same time, Mozilla is about flattening power for the benefit of the individual and the betterment of the web. We could do really well if we purposely reach further outside the "top" 20% of web developers.
But what would it look like? Does Mozilla send staff to smaller local events where we already have remote employees? Do we sponsor these events? Do we bring some developers from small communities to Mozilla HQ for a work-week? I don't know - I'm asking.
bugzilla-agile
I'm not a pure Agilista™ but we're always trying to improve our development process for MDN. John and I like Scrum and XP stuff, but every team does Agile™ a little differently - as they should. A while back we shopped around for tracker tools and stuck (as Mozillians always do) with bugzilla. We - i.e., mostly John and Jay - also looked at acunote, planbox, scrumdo, agilezen, pivotal, and agilebench.
What we most like about Bugzilla:
- Bugzilla is Mozillian - it channels the work of tens of thousands of Mozillians; we can cc anyone in the community on a bug
- Bugzilla is open - we can link anyone in the world to a bug
- Bugzilla is versatile - as Jonath says in Bugzilla for Humans, it's the devil we know
On the latter point, I've forked some agile features onto Greg's BugzillaJS to help us work more Scrummy™. Our most pressing issue is managing releases - our scope keeps bloating and our releases keep slipping. So we're starting to use the Agile/XP concept of "points" to estimate bugs, track our team velocity, and hopefully improve our release rhythm and reliability. Behold our improved "sprint backlog":
There's a few new things going on here. Here's a summary of how we're doing what we're doing:
Milestone releases - We use the milestone field of bugzilla for our releases. Next up is 0.9.9 scheduled for August 2nd release. As we move to a more continuous development and release cycle, the milestone version numbers lose meaning (just like Firefox), but we want to track releases. (Sorry James, we're not deploying every check-in just yet).
Whiteboard overloading - We're using a tag=value pattern in the whiteboard to add new "fields" because adding fields and values to bugzilla requires IT changes and they're over-worked as it is. In our case,
- 'u' - the primary user of the feature/bug (faster and more programmable than writing "As a ___, I want ___" every time
- 'c' - the component that the feature/bug modifies
- 'p' - points
Calculating points and stories - For any search that includes the "whiteboard" column with the specified tokens, the addon sums the number of "Open" and "Closed" stories and points for the release.
Pretty graphs - data visualization FTW. Seriously, graphs give us a quick snapshot of the open v. closed bugs, and in which component we're spending our effort. This is important for MDN because we want to re-write the wiki while continuing to deploy site enhancements and changes. Now we can see exactly how much of our effort is going to the wiki as compared to other components.
I really hope this improves our releases and makes life easier for devs. Points by release and components are our most pressing needs so we can set realistic release and product expectations, and keep ourselves honest about where we're spending our effort. We'll probably add velocity and burndown charts once we finish a few point-based releases.
If anyone else wants to use it, my fork of BugzillaJS is up on github; download the .xpi file and open it with Firefox to install the addon. Feedback and pull requests are very welcome!
Edit: I should point out we're inspired by other agile Bugzilla tools - the Songbird team has wrestled bugzilla into their Agile process, and fligtar created moxie to help AMO product management.
Webdev Offsite & Open Source Bridge
Back when we released Firefox 4, I remember a fellow mozillian tweeted
The folks I work with are built out of brains, grit, guts and glory. Amazing. I am _so_ lucky.
Last week I verified the sentiment when I met a bunch of the folks I work with - many for the first time. Our WebDev offsite (photos) was great. We had a nice little Webdev unconference at the Urban Airship office on Monday, then we all attended, and a few of us spoke at, Open Source Bridge.
On Friday a bunch of us hacked on little projects and, as the conference wound down, we snuck into a room for a hack-n-tell. It showed me how much skill we have in webdev, and our evenings together showed me our group's passion. I'm humbled by both.
