David Golden - Cooking Perl with Chef

Title: Cooking Perl with Chef

Name: David Golden

Grant Manager: Alan Haggai Alavi

Duration: 3 months

Approved: February 2012

Synopsis
I propose to create ready-to-use tools and supporting tutorials that teach Perl developers (and the sysadmins that support them) how to deploy Perl applications repeatable and reliably using the Chef configuration management tool.

Benefits to the Perl Community

  • First-class support for Perl in Chef, just like other web development languages
  • Open cookbooks and tutorials to encourage sharing techniques and practices for app deployment

Deliverables

Code deliverables:

  • Publish a Chef cookbook for Perl interpreter deployment
  • Publish a Chef cookbook for CPAN module deployment
  • Publish a Chef a cookbook for Plack application deployment
  • Improve the Pantry command line tool for configuring servers with Chef Solo
    • Add command-line configuration of "nodes" (servers) and "roles" (configuration patterns)
    • Add support for multiple environments (e.g. "dev", "test", "production") with different configuration options per environment

Note: in Chef a "cookbook" encapsulates all code and metadata needed to automate configuration of particular resources. C.f. About Cookbooks (Opscode wiki)

Teaching deliverables:

  • Publish a presentation-style introduction to using Perl with Chef
  • Publish a document-style "hello world" tutorial with step by step instructions for deploying a simple Plack app
  • Publish a document-style "real world" tutorial with step by step instructions for deploying a complete application stack for a real application
  • Release a video demonstration of "hello world" deployment, as either a screencast or a recording of a conference/seminar presentation

Project Details

Motivation (aka "the problem")

Automated configuration management tools are increasingly part of the toolbox for Internet technology companies, but support for Perl and Perl application frameworks lags behind other dynamic languages. This makes Perl harder to integrate into an environment that already uses such tools and makes it harder for Perl-oriented companies to adopt such tools.

While these difficulties can be overcome, deploying Perl applications with configuration management tools remains an idiosyncratic black art, with the knowledge of how to do it locked up in the minds (or companies) that have done it. For someone new to configuration management, the learning curve is steep.

Approach (aka "the solution")

Over the last couple months, I have been experimenting with deploying Perl applications using the Chef configuration management tool. I demonstrated my "proof-of-concept" code at the Orlando Perl Workshop in January. I continue to tinker with this code and seek a grant to support and motivate turning this experiment into public resources for the Perl community at large.

Using configuration management tools for Perl applications requires addressing several issues that go beyond what typical Perl/CPAN toolchain programs deal with, such as:

  • Idempotent deployment automation
  • Versioned library dependency management
  • Application restart when configuration (including dependencies) changes

Multiple perls and multiple apps deployed simultaneously on a single serverMy proof-of-concept combines existing tools like perlbrew, local::lib, carton, Plack, and so on to achieve these goals. While the proof-of-concept tools work, they need further refinement, documentation end testing before they are release-ready for general use.

Once these cookbooks and tools are created, I want to make them easy to learn. Since not everyone learns the RTFM way, I propose to create two step-by-step tutorials that walk someone though deploying an application from bare (virtual) metal to live server.

The first tutorial will be the configuration management equivalent of "hello world" -- it will be the smallest code that can show the essential components and steps of the solution. Since not everyone learns from reading tutorials and since there appears to be popular appeal for screencast tutorials, I will also record a video of this tutorial. (Or, if an appropriate conference/seminar opportunity arises to make such a demonstration during the grant period, I will provide that instead.)

The second tutorial will be a "real world" one that shows how a full application -- including supporting, non-Perl components like a web-server, database, caching layer, etc. -- can be deployed and configured. I have not settled on an application for this tutorial, but I consider the CPAN Testers Metabase or the Jitterbug continuous integration server to be leading candidates.

Inch-stones

See "Deliverables" for general work-blocks.

Each of the code deliverables will be broken into "coding", "documentation" and "release" inch-stones.

Written teaching documents will be broken into "rough draft", "peer review", "final draft" and "release" inch-stones.

The video deliverable will be broken into "storyboard", "recording", "post-production" and "release" inch-stones.

Project Schedule

The project is estimated to take three months. I will begin immediately upon being awarded the grant.

Completeness Criteria

All code and teaching deliverables will be published online under open licenses.

  • Cookbooks will be published to the Chef Community Cookbooks site (which is like CPAN for Chef).
  • Pantry will be published to CPAN.
  • Tutorials will be published either to my blog or to an appropriate Perl tutorials site.
  • Video will be published to YouTube and/or Perl-specific video sites.
  • Code, presentations and documents will also be made available via Github for community access and improvement.

Bio

I have a long history contributing to the Perl 5 community:

  • Active Perl 5 Porter responsible for two dev releases, several new features and the occasional bug fix.
  • Developer of the first several releases of Strawberry Perl.
  • Prolific CPAN author. Contributor to or co-maintainer of a large subset of the CPAN toolchain.
  • Leader of the CPAN Meta Spec v2 revision process
  • Lead architect and manager of the CPAN Testers Metabase that collects nearly a million CPAN Testers reports every month.
  • Moderately-frequent blogger and conference/seminar presenter on Perl-related topics.