Let's Encrypt on Heroku with DNS Domain Validation

We needed to renew and update our certificate for www.codesy.io, and I’ve been wanting to use Let’s Encrypt for a while. I had read and tried some other guides for using Let’s Encrypt on Heroku, but none of them cover DNS domain validation. The steps are roughly:

  1. Install certbot
  2. Use certbot to generate a manual cert
  3. Deploy a TXT record to your DNS
  4. Upload signed certificate to Heroku
  5. Update your DNS Target

Install certbot

First, you’ll need certbot:

brew install certbot

Note: The certbot site contains install instructions for other systems.

Use certbot to generate a manual cert

With certbot you will need to generate a cert to manually install to the Heroku server, and specify DNS as your preferred challenge:

sudo certbot certonly --manual --preferred-challenges dns

Note: certbot needs sudo to put resulting files into /etc/

certbot will ask you the domain for which you want a certificate …

Certbot Domain Prompt Screenshot

… and if you’re OK with your IP being logged as having requested the certificate …

Certbot IP Prompt Screenshot

… and will finally tell you what DNS TXT record to deploy:

Please deploy a DNS TXT record under the name
_acme-challenge.www.codesy.io with the following value:


Once this is deployed,
Press ENTER to continue

Note: Don’t press ENTER until you have deployed your TXT record

Deploy a TXT record to your DNS

Your domain registrar likely has its own docs for adding a TXT record. Here are some links to a few:

Upload signed certificate to Heroku

Go back to certbot and press ENTER. It will create signed certificate files in your /etc/letsencrypt directory.

SSL is now included on all paid dynos on Heroku. The $7/mo. for a hobby dyno is still cheaper than $20/mo. for the old SSL Endpoint add-on. So, to change to a hobby dyno, go to your app’s Resources panel and click “Change…”

Heroku Dyno Resources Screenshot

Then, use heroku certs:add to add your Let’s Encrypt fullchain and privkey files.

sudo heroku certs:add --type=sni /etc/letsencrypt/live/www.codesy.io/fullchain.pem /etc/letsencrypt/live/www.codesy.io/privkey.pem

Note: Again, heroku needs sudo to access files in /etc/

You can also copy+paste your certificates’ contents in your app’s settings dashboard - under “Domains and certificates”, click “Configure SSL”.

Update your DNS Target

Finally, update your DNS CNAME record for your domain to point to the certificate-domain.herokudns.com. In our case, it was www.codesy.io.herokudns.com

Hover DNS Screenshot

Enjoy your Let’s Encrypt-verified site!

Codesy Verified Screenshot

Take 5 minutes to help web developers and earn Stack Overflow rep

Update 2014-02-28: Gareth helped me make this dynamic spreadsheet that auto-updates from Google Analytics every night.

tl;dr - go thru Today's stackoverflow MDN links to fix and fix them in the StackOverflow answers.

Stack Overflow Rep

Helping developers is my professional and personal passion (See Exhibits A, B, C, and D). MDN and Stack Overflow are both great resources for web developers. But following a Stack Overflow answer link to a 404 page is frustrating and dis-heartening.

xkcd: Wisdom of the AncientsI added GA event tracking for 404's on MDN. I originally wanted to help writers decide which wiki pages to prioritize, based on how many times an MDN reader clicks an internal link that results in a 404. But when you record metrics and analyze them, you find the unknown unknowns, and "it’s the unknown unknowns that really matter, because that’s where the magic comes from."

So, I learned that the vast majority of 404's on MDN are from external links. (Duh!) And the biggest single source of those are old Stack Overflow links. Luckily, Stack Overflow allows us to edit those answers to update the links to MDN. There's even a special "Excavator" badge for editing a post that's been inactive for 6 months.

So, if you've got a few minutes, you could help clean some of these links up:

  1. Look thru the Google Spreadsheet; there's a long tail of links to fix
  2. Use our excellent new MDN search system to find the right doc for the link
  3. Edit the answer and earn +2 rep!

Note: Sometimes we may want to create REDIRECTs for the 404 pages to help other visitors, so if you fix some links, please mention it to us on this thread.

screen recording green

Both QuickTime Screen Recording and iShowU kept giving me a green video until I disconnected my external monitor. I got the idea from this Apple Support thread comment to disable the external graphics chip. There are other ideas in the full thread.

Hopefully this post helps the next person find their answer faster than I found mine.

MaaS: MDN as a Startup

tl;dr – I want to reboot MDN as a Lean Startup – a platform for web developer knowledge and community. There's a bunch of new kinds of work to do. Go to the MDN Metrics project proposal on wiki.moz, join the thread on mdn-drivers, or comment here to give feedback.

After I read Kanban and The Phoenix Project I picked up Lean Startup and Lean Analytics. I love that the Lean Startup community is inclusive of intrapreneurhsip – people making new products inside larger host companies. The “Developer Program” we've discussed at Mozilla is a perfect example of intrapreneurship, and we should treat it as a User-Generated Content startup.

We're already doing this ...

We're redesigning MDN and already launched the redesign to beta testers. It looks great - it highlights important content, improves search and discover-ability, and it's easier to read article text. We're improving the redesign with quick continuous deployments to beta users; we even pushed to production 4 times during Mozilla Summit - 2 of which were changes made by new contributors from our impromptu MDN hack night!

But ...

We need to measure

It's no secret that the redesign project has been stressful across all teams - Product, UX, Creative, Development, Content, and our community. Pressure can be healthy - constraints foster creativity, innovation, and some pretty epic hacks. But pressure without purpose is just wasteful stress. And the driving philosophy of Lean is to eliminate inefficient and wasteful activity.

Dan McKinley from Etsy gave a very insightful talk about Designing for Continuous Experimentation at Warmgun design conference. He tells the story of Etsy implementing an infinite scroll project:

So when we decided to do this we just went for it. We designed and built the feature, and then we figured we'd release it and it'd be great.


Eventually we came to terms with the fact that infinite scroll had made the product worse, and we had changed too many things in the process to have any clue which was the culprit.


So if we go back to our "product plan," we see a couple of major things wrong with it. We did a lot of work and it was pointless. A better way to have done this would have been to validate those premises ahead of time and then make the call.

(Emphasis mine) So, I'm pushing hard for MDN to validate premises before we expend time, effort, and stress on projects.

What should we measure?

MDN is site for user-generated content like Wikipedia, reddit, Twitter, Facebook, etc. Lean Analytics has an entire chapter devoted to UGC metrics for:

  • Visitor Engagement - how often do our readers come back, and how long do they stay?
  • Engagement Funnel and Changes - how many users register; how deeply are they involved with MDN and how do we encourage them to deeper activities?
  • Notification Effectiveness - how many newsletter subscribers and twitter followers do we have? How many of them open the email or click thru to the links?
  • Content Creation and Interaction - how many people tag, edit, translate, or review articles; how many comment or create demos?
  • Value of Created Content - how visitors read certain topics and articles; which traffic sources bring more engaged users?
  • Content Sharing & Virality - how many users tweet, like, or otherwise share articles; how do the recipients behave as a result?

For example, I've been digging into our MDN Engagement Conversion Funnel:

MDN Engagement ChartFor me, the engagement cliff between visiting and creating an account is too huge. So that measurement tells me we should add more activity features between visiting and creating an account. Maybe something like:

MDN Potential Engagement Chart

If we add features, we should implement minimum viable features to validate our assumptions about the impact we want - e.g., does a sharing or voting feature actually increase account registrations and editing behavior?  So we'll also want to define "the impact we want" - higher quality content, more visitors, etc.

How should we measure?

So far I've done all this by hand, but I'm proposing that we start a project for MDN Metrics on the site itself. Something similar to the interface of SUMO's KPI dashboards, though our KPI's will be different. That way everyone in the MDN community has the access to our metrics when discussing MDN product and priorities. If you've read this far and want to give feedback, leave comments here, or join the thread on mdn-drivers.

team fortress 2 enable steam community ingame

Another quick google-search-turned-post to help others looking for a solution to this problem.

After my last Steam upgrade (Mac client) the community overlay stopped working in Team Fortress 2. This breaks a number of features, including the "Invite" button when creating a party. This is the solution that worked for me:

1. In Steam Library, right-click Team Fortress 2 and choose Properties
2. UN-CHECK "Enable Steam Community In-Game"
3. Quit Steam
4. Re-open Steam
5. RE-CHECK "Enable Steam Community In-Game"

Overlay starts working again!

HTML5 App Stores don't go too far enough

tl;dr - HTML5 app stores don't leverage the web as much as we should.

Futurama Political Debate

 I'm convinced that packaging HTML5 apps for stores is emulating failure. There are lots of reactions. Among the comments here, Les made the opposite assertion of mine - "where app stores win over plain old URLs is discoverability. ... URLs are precise, but they’re sharp and pokey for most people to handle."

HTML5 App search engine > Pokey URLs

Les made his own counter-point that we fixed pokey URL's with search engines. So, do we need "a special case search engine where the results are constrained [to HTML5 apps]"?


The "webby" approach is to crawl the web to build an index of all app manifests. But how can we discover the manifests? None of the HTML5 app stores document a way to hyperlink manifests.

But maybe we can use schema.org SoftwareApplication type, with its convenient installUrl property. We can even use the WebApplication sub-type's 'browserRequirements' property to designate a Firefox, Chrome, or W3C manifest. So Incredible Pizza could expose their HTML5 app like so:

<div itemscope itemtype="http://schema.org/WebApplication">
  <a itemprop="installUrl" href="http://app.incrediblepizza.com/incrediblepizza.webapp"><span itemprop="name">Incredible Pizza</span> for <span itemprop="browserRequirements">Firefox</span></a>
  <a itemprop="installUrl" href="http://app.incrediblepizza.com/incrediblepizza.crx"><span itemprop="name">Incredible Pizza</span> for <span itemprop="browserRequirements">Chrome</span></a>
  <a itemprop="installUrl" href="http://app.incrediblepizza.com/incrediblepizza.wgt"><span itemprop="name">Incredible Pizza</span> for <span itemprop="browserRequirements">Opera, Windows Phone</span></a>

Voilà. We have web technology to make an HTML5 App search engine. But why are HTML5 app stores making developers manually submit manifest URLs to directories when we could leverage hyperlinks - the central nervous system of the web? As a bonus, a hyperlink-fueled HTML5 app search engine is complimentary to existing mobile platforms. This could be nice:

iOS App Search mockup

Apps contextualize content

Still, an HTML5 app search engine doesn't leverage the web as much as we could. Ambrose Little suggests "discoverability is worse [on the web] because it’s not just apps, it is all kinds of content, much of which is not pertinent nor optimized for your device." This gets to the crux of the issue: outside of pure experience apps like games, when we say we want apps, what we really want is relevant, contextual, optimized content on our mobile devices. Can we deliver relevant, contextual, optimized content to mobile devices without apps?

In Programming the Mobile Web, Maximiliano Firtman touches on this in his chapter about geolocation:

One of the great features of mobile devices is that they can go everywhere with us. That is why the where is very important context to be considered by our websites. Knowing the user's location can help us to show useful contextual information -Firtman

Geolocation is a great place to start, since it's so widely supported on existing mobile browsers. I suggested that incrediblepizza.com could return location-specific content using a captive wifi portal or geolocation. But fuzzy search can also deliver location-specific content, as Google Maps does; though it could be improved to deep-link into a domains' pages with something like GeoURL or a geofenced sitemap.

I.e., if incrediblepizza.com added:

<meta name="geo.position" content="36.05877;-95.8831" />
<meta name="ICBM" content="36.05877, -95.8831" />

to their Tulsa store location page, and maybe add a geo point and radius to their sitemap:

<url xmlns:geo="http://geositemaps.org">

mobile search could do something like this:

iOS Search for Pizza

Search already works

We can search for a pizza place nearby. So what? Foursquare already does that.

But do we have to search for it? As predicted by Mike Masnick back during Yahoo's misguided deal with Microsoft in 2009, "People are discovering that information finds them, rather than them going in search of information. Search already works. The next interesting challenge is in improving the way information finds you, rather than the way you find information."

Improving the way information finds you is how social competes with search, and it's the way mobile web should compete with app stores. Location, time, personal information, preferences, friends, actions - all of these create the context in which information should find us. Chris Cox from Facebook summed it up:

[Mobile phones] should be contextually aware. Check-in to flights, find deals at grocery stores, etc. These things take a bunch of clicks now — it’s all wasting time. The phone should know what we want.

My phone should know:

  1. I have 2 kids under the age of 4
  2. We're near south Tulsa
  3. We typically eat out on Tuesday nights
  4. It's almost dinner-time

It should query the web. To which incrediblepizza.com replies:

  1. Kids 3 and under eat free
  2. We're located in south Tulsa
  3. We have double-play-points specials on Tuesdays

incrediblepizza.com and my phone inform me automatically. This touches semantic web stuff; and maybe that's something we mobile web developers should be touching. Because ...

The only group who can contextualize everything is everyone

The web will obsolete app stores ... again. In Scott Jenson's "Why Mobile Apps Must Die" talk at BDConf 2011, he longs for a web-based phone, and a discovery service that helps us find the right needles in the needle-stack of connected things. He calls it "a baby step towards this idea of doing just-in-time interaction" and says "I can guarantee you this will not happen thru apps."

Why not?

Christian reminds us of Yahoo's first (failed) attempt at re-creating a categorized directory of the web. If you don't have time to read Clay Shirky's classic perspective on ontology at least watch this great related 5-minute video:

Even if you skipped both, here's the crux of the matter (emphasis mine):

Yahoo ... added the shelf back. ... The charitable explanation for this is that they thought of this ... as their job, and as something their users would value. The uncharitable explanation is that they thought there was business value in determining the view the user would have to adopt to use the system. Both of those explanations may have been true at different times and in different measures, but the effect was to override the users' sense of where things ought to be, and to insist on the Yahoo view instead.

You can replace "Yahoo" with "Apple" in the preceding statement and it mirrors the current situation of Apps. The web out-grew Yahoo's shelf; it will out-grow Apple's, Google's, Microsoft's, and Mozilla's shelves. Our sense of where things ought to be and how things ought to find us is better than any vendor's. "The web is going to win again. The web always wins." as Johnath says.

The web will win sooner if we spend less time on the shelves and more time delivering relevant, contextual, optimized content to mobile devices.

Packaged HTML5 Apps: Are we emulating failure?

tl;dr - URLs delivered a better experience than native desktop apps; they can do the same for mobile apps.

update: see my follow-up post - HTML5 App Stores don't go too far enough.

HTML5 in a box

When we discussed Sencha's Fastbook app on the mozilla.engagement.developers list Robert pointed out that Sencha compared their HTML5 app running in mobile web browser to Facebook's HTML5 app running in an iOS WebView - the way Sencha & PhoneGap packaged HTML5 apps actually run. It's widely discussed, as we did with Fastbook, that HTML5 apps perform worse in iOS UIWebView than iOS Safari. There are similar stories about Android WebView performance, and Tulsa Web Dev and HTML5 PhoneGap developer Rod just posted a similar performance-related anecdote about BlackBerry WebWorks.

That made me think, "PhoneGap is just that - a stop-gap." ... "we should drop this whole PhoneGap App nonsense and go (back) to mobile web at simple URL's" since:

  1. App discover-ability & installation sucks
  2. App content discover-ability sucks

App discover-ability & installation sucks

A couple months ago, my wife and I took our daughters to Incredible Pizza. When we sat down, I saw this table-tent:

Incredible Pizza Table-Tent

I spent a couple minutes trying to make this "check-in" UX work on my wife's phone. The table-tent literally reads: "TIP: A QR-Reader App is necessary to scan QR codes. There are several free ones available. Check your app store." So, here's what I did to find an app:

  1. Connect to Incredible Pizza wifi
  2. Tap "All apps"
  3. Swipe down to "Play Store"
  4. Tap "Play Store"
  5. Tap "Search"
  6. Type "qr reader"
  7. Swipe thru a huge list of apps
  8. Pick one at random (*)
  9. Tap "Install"
  10. Tap "Accept & Download"
  11. Wait for download & install

Android prompts me with a wall of permissions that I have to accept in order to install the app. And because app updates with new permissions can't be automatically downloaded & installed, many app developers just make their app request all permissions.

* Especially interesting to developers - I picked a free qr reader app at random, verifying that App Stores are more like casinos than gold-mines.

Content discover-ability sucks

Once I had the QR reader app, I started to use it to check-in to Incredible Pizza on Facebook:

  1. Hold the phone in front of the card
  2. Tap "Visit Link"
  3. Tap "Browser"
  4. Go thru 5 redirects to a prompt to open native Facebook app
  5. Facebook opens
  6. Wonder: "Why am I trying to do this in the first place?"

There was no human-readable URL on the card. I had to choose between "Browser" and "VZ Navigator" that came with the phone from Verizon. The redirects took about 20 seconds. Finally, since they don't deep link into the native app, I ultimately end up just staring at the Facebook app as if I'd opened it myself - nothing there at all about Incredible Pizza.

This totally obscures the app's content, and value, from the user. If I package my HTML5 app with PhoneGap I have to also register a custom URL scheme to link to a specific part of it - something PhoneGap doesn't help you to do(?). And even when you do it, you will need to make and deploy an update to the app anytime you want to add new URL-driven logic to it, invoking another round of app review.

The Plain Old URL approach

Contrast the above experience with a more traditional web approach:

  1. Connect to Incredible Pizza wifi
  2. Tap Browser
  3. (Maybe) Type "incrediblepizza.com"
  4. (Maybe) Tap "Yes" at a geolocation prompt

Incredible Pizza could immediately give me the store-specific web page via captive wifi portal with big buttons for location-and-date-specific coupons, facebook check-in, etc. Or, if I'm not on their wifi, I type "incrediblepizza.com" to get the same thing via geolocation. In addition to a simpler experience, deploying new or updated content is as simple as updating the website.

So why package HTML5 apps?

Considering all the above, I usually ask web devs using PhoneGap: "Why are you using PhoneGap instead of just putting an HTML5 app at a URL?" and they often reply "everyone wants an app." This "everyone wants an app" situation is a problem we developers and designers should address - but not just technically. We should work with our clients and customers to consider the full user experience of native vs. web - especially app installation & content discover-ability.

... although many brands and creative agencies believe that they need to develop iPhone apps, what they need in reality is a good mobile site. It costs less to develop, manage and work across all handsets.

- HTML5 - The End of Apps | Mobile Europe

If you're making an HTML5 app, consider - do you want to make a native desktop application? Why or why not? Then consider if the same reasoning is true for the native mobile application. Consider the URL experience for your customers. Walk your clients thru both native app and URL experiences on mobile devices. We used web URLs to deliver a better experience than native desktop apps - we should do the same with mobile apps.

Good HTML5 Packaging - URL Bookmarks

Vendors like Mozilla (disclaimer: my employer) and Google are also working on manifests to install HTML5 apps on Firefox (OS) and Chrome (OS). Opera and previously(?) Microsoft (via Silverlight) use the W3C Packaged Web Apps (Widgets) standard. I like these efforts if they contribute towards a consistent "installation" experience that works more like an enhanced bookmark. I.e., turn this:

  1. Visit URL
  2. Tap "Add to ..."
  3. Tap "Add to Home Screen"
  4. Tap "Add"

into this:

  1. Visit URL
  2. Tap "Add to Home Screen."

But next time you make an app, consider the tried-and-true web experience:

  1. Tap Browser
  2. Type URL



Mozilla & Tulsa Web Devs

tl;dr - Mozilla sponsored Tulsa Web Devs for all of 2012. This year Tulsa Web Devs:

  • Grew to 215 members
  • Facilitated 3 spin-off groups
  • Had 11 monthly meetings
  • Had 50 weekly coworking days
  • Had 4 hack days
  • Ran 2 web tracks at local conferences
  • Hosted 2 hackathons
  • Participated in 1 Startup Weekend
  • Helped draft 1 City Council Resolution
  • Worked on 32 open-source web projects

I'd also like to see what other Mozilla remotees are doing in their local web communities ... ?


We've added a bunch of great new members this year. I won't name any specific names for fear of forgetting someone, but our members have started spin-off groups like Red Dirt Web Designers, Red Dirt Graphic Designers, and Tulsa DrupalDevs. We also have a number of WordPress designers and dev's - though they haven't made their own sub-group yet. But our members have presented at OKC WordPress meetup and OKC.js May and June meetings.


Our average meeting attendance jumped when we moved from Fab Lab Tulsa to i2E location downtown. We went from ~12 in January to ~30 in November.

By far my favorite meeting was our August meeting where we asked anyone and everyone to simply answer the question "What have you been working on?" in 5 minutes or less where the lightning talks ranged from Couch to django to WordPress to Node.js. I hope to do more meetings like this; even if we pick a single topic, I'd love to have multiple presenters for it.


This year we finally - consistently - hosted (at and with Fab Lab Tulsa) coworking days (almost) every Friday. Attendance is anywhere from 1-2 to 10-12, with notable spikes anytime to dev's from consumeraffairs.com make the field trip. ;)

Many of our open-source project ideas come up during coworking days, and we give each other all kinds of technical advice - some of us have saved dozens of hours of work by simply asking a tech question around the tables.

Hack Days

Our hack days typically involved or included working on our open-source projects to prepare for other events. We started the year strong with Hack Days - doing one every-other-month to set up for our Spring Hackathon - but then we ran out of steam. Probably due to ...

~Web Tracks

This year we organized content for 2 "(sorta) web tracks" at established local technology events - a Gov2.0 track at Tulsa School of Dev, and an HTML5 track at Tulsa Tech Fest.

Our Gov2.0 track was pretty much a flop - it's not our area of expertise, the overall event had poor promotion, and the event logistics were terrible.

The HTML5 track at Tulsa Tech Fest was really good. I got Yury to come over from OKC to speak, Patrick to give a fresh edition of his CSS talk from the previous year, and even got Olivier Bloch to speak about HTML5 for mobile cross-platform app development.

Both of our tracks lead into ...


Our Spring Hackathon focused on Gov2.0 and "open data" projects. In 2011 we - especially John Whitlock - made a couple of very valuable and noticeable "open data" projects: TRIF and the Tulsa Transit GTFS project. Motivated by their success, we tried to do more of the same type of projects. We also increased the duration from 24 to 48 hours. We made some cool stuff, but, IMO, the format was too long and the domain was too narrow.

Our 2nd annual Fall Hackathon snapped back to 24 hours and focused only on "apps" - any app would do. Mobile web, HTML5, iOS, Android, Windows Phone, whatever. We did *some* promotion but not so much to be stressful, and we still had a good turnout and made some good projects and progress. I think it was just about the Right Size and Scope™ to be a sustainable yearly event.

Startup Weekend

A bunch of Tulsa Web Devs participated in this year's Tulsa Startup Weekend, creating projects like TFDD.co, Wallcade, and Picirus (this year's winner!). I also started a GeoNotes app aimed at Firefox OS, but our team was a bunch of backend dev's trying to learn HTML, CSS, and JavaScript so I wasn't confident enough to even show it at the closing ceremony.

City Council Resolution

Our Gov2.0 work attracted lots of attention from local civic interest groups, including Transit Matters, Tulsa Now, Tulsa Young Professionals, and eventually the Tulsa City Council itself. So, one of our city councilors contacted me about how to encourage and support the kind of projects and work we're doing. I suggested we should pass a City of Tulsa Open Source Resolution like some I had heard about from Portland, San Francisco, and Vancouver. It's still in draft stage but we hope to get it on a city council meeting agenda sometime next year.


We made too many projects to list them all, so just check out our Tulsa Web Devs GitHub org page or our Projects page. Our projects have spanned from local consumer interest to irc bots to wordpress plugins to open data.


What does all this have to do with Mozilla? In running the group, I've tried hard to NOT make it all about Mozilla, but Tulsa Web Devs now boasts 1 Mozilla Contributor & 1 Mozilla Rep, 2 Open Web Apps, and dozens of engaged Firefox users. Now many members have asked me to bring MORE Mozilla activity into the group - to host a Mozilla hack day, bring speakers in from Mozilla, and help people contribute to Mozilla's websites. I'm looking forward to more collaboration between Mozilla & Tulsa Web Devs in 2013!

I'd also like to see what other Mozilla remotees are doing in their local web communities. Got a good story to share?

Home / groovecoder by groovecoder is licensed under a Creative Commons Attribution-ShareAlike CC BY-SA
Home / groovecoder by is licensed under a Creative Commons Attribution-ShareAlike CC BY-SA