Perl 6 Roadmap to helping with Development

Note: This guide is a draft work in progress. I myself develop locally on my Vista laptop and desktops and publish to my Linux servers. So this guide describes how to do things on Vista. It would be good if someone could supplement the guide with Linux examples.

When I decided to help with Perl 6 development I didn't know where to begin. I thought it'd be useful to other people if I documented the things I learned along the way into a guide.

Background: Things you need

The two main Perl 6 implementations are Pugs and Rakudo.

Pugs (Perl 6 users golfing system) is built on Haskell

Rakudo is build on parrot, the VM that was designed for it and any scripting language

Pugs site is at:-
http://www.pugscode.org/

The latest development code is in a subversion repository at:-
http://svn.pugscode.org/pugs/

Rakudo is at:-
http://www.rakudo.org

The latest development code is in a github repository at:-
http://github.com/rakudo/rakudo/tree/master

Pugs came about well before Rakudo while Parrot was in initial development
Due to this, and Rakudo not wanting to duplicate things, the main specification test, documents, etc are in the Pugs repository only.

So you are going to need both subversion (SVN) and git so you can easily keep up to date with the latest development release. Both programs have lots of features and functionality. I'm only going to cover the bits you need.

Installing Subversion

SVN is available from http://subversion.tigris.org/

I'm running Vista so I downloaded (depending on how old this guide is, the version number you see might be a lot newer):-

svn-win32-1.6.0.zip
svn-win32-1.6.0_dev.zip
svn-win32-1.6.0_pl.zip
from:-
http://subversion.tigris.org/servlets/ProjectDocumentList?folderID=8100

And unzipped them all to:-
C:\Program Files\svn
Which created the folder
C:\Program Files\svn\svn-win32-1.6.0

I then added C:\Program Files\svn\svn-win32-1.6.0\bin to my PATH.
(Update your path environment variable to include this folder on the end. In Vista32 in go Start->(Right click on)Computer->Properties->Advanced System Settings, then click the Advanced tab at the top of the window, then the Environment Variables button near the bottom. Finally under 'System Variables' scroll to find Path, select then click the edit button. Scroll to the end of the list and add ";C:\Program Files\svn\svn-win32-1.6.0\bin". Click Ok and Ok again. You may need to reboot for this to take full effect.)

Installing git

You'll want a github account. Visit http://www.github.com to signup for free.

Github provides a guide for installing and using git on windows
http://github.com/guides/using-git-and-github-for-the-windows-for-newbies

You can follow that or my version is here. For windows there is a version called msysgit available from
http://code.google.com/p/msysgit/downloads/list
At this time all versions are beta, so I grabbed the latest Git-1.6.2.2-preview20090408.exe
I ran the installer, selecting 'C:\Program Files\git', 'Use Git Bash only' and 'Unix style line endings'.

Rather than using the command prompt like with SVN, this version of Git will install a separate Linux style Bash to use.

Git wants you to have your own SSH-key. Open Git (Start->All Programs->Git->Git Bash). type:-
ssh-keygen -C "your@email.com" -t rsa
Swapping your@email.com for your email address. It'll prompt you to 'Enter file in which to save the key ...' Just hit enter for the default. Then it'll ask for a password and confirmation.
To be able to push your changes back onto github you'll need to save your public ssh key in your github settings. Login to github, click on 'account' (top right). There is a box titled "SSH Public Keys", click on add. Now you need to copy and paste the key you created into the box. Open the file C:\Users\YOURUSERNAME\.ssh\id_rsa.pub with a text editor (notepad), select all, copy and then paste it into github. I called mine main key. Then click 'Add Key'.

You'll also what git to remember who you are. To do this you need to set some global config with:-
git config --global user.name "YOURUSERNAME"
git config --global user.email "your@emailaddress.com"

Installing MinGW (gcc compiler)

You'll need to be able to compile parrot which is in c. On Linux your probably already have gcc installed, if not you can grab a package from your distro. On windows gcc is packaged as MinGW and available from:-
http://www.mingw.org/wiki/MinGW

I downloaded the Automated MinGW Installer and ran it selecting a full install to c:\MinGW.
http://sourceforge.net/project/showfiles.php?group_id=2435

Then update your PATH to include c:\MinGW\bin (same process as we did for SVN).

It's probably a good idea to also download the GNU Source-Level Debugger from MinGW's downloads page. I grabbed gdb-6.8-mingw-3.tar.bz2 and extracted to c:\MinGW with winrar.

Installing MSYS (windows versions of bash, etc)

MSYS is part of the MingGW project and can be downloaded from their site:-
http://sourceforge.net/project/showfiles.php?group_id=2435
I found the download options a bit confusing. You need to click on "MSYS Base System", then msysCORE-1.0.11-20080826.tar.gz.

Extract to C:\msys
The from command prompt run
cd \msys\postinstall
pi.bat

This runs the postinstall script that ties MSYS in with MinGW. The options I gave were:-
y
c:/mingw

I updated my Windows path to include c:\msys

Installing Haskell

Pugs is built in Haskell, so you'll need the Haskell compiler to build it.
http://www.haskell.org/ghc/download.html
Note: At time of writing there is an issue with Haskell v 6.10.x (http://hackage.haskell.org/trac/ghc/ticket/3162), I had to redo my whole pugs build with Haskell 6.8.3 from
http://www.haskell.org/ghc/dist/6.8.3/ghc-6.8.3-i386-windows.exe

I ran the installer and installed to C:\ghc\ghc-6.8.3

Installing Perl 5

You'll need Perl 5 but I'll assume you have that already. I'm using ActivePerl 5.10

Installing IIS

Not all copies on Windows come with IIS, if yours doesn't then skip to installing Apache.

Vista home premium and above

From the Control Panel select "Programs and Features". On the left there is a link to "Turn Windows features on or off". Check the box for "Internet Information Services", if you get a blue square rather than a tick, expand the option and manually select all the sub options. Click on ok, you may need your Vista CD to complete this operation.

Once installed you can test by opening your browser and typing the address "localhost".

Installing Apache

Download the latest Apache from:-
http://httpd.apache.org/download.cgi

For me this was Apache 2.2.11, I grabbed the windows installer with openssl apache_2.2.11-win32-x86-openssl-0.9.8i.msi

I installed to the default place as a service on port 80, which conflicted with IIS, but I'll changed that in the config after. I'm also going to move the htdocs and cgi-bin folders and enable cgi scripts in all folders.
Open C:\Program Files\Apache Software Foundation\Apache2.2\conf\httpd.conf
Change the lines:-
Listen 80
to Listen 8080
DocumentRoot "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs"
to DocumentRoot "C:/htdocs"
<Directory "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs">
to <Directory "C:/htdocs">
Options None
to Options Indexes Includes FollowSymLinks ExecCGI
<Directory "C:/Program Files/Apache Software Foundation/Apache2.2/cgi-bin">
to <Directory "C:/htdocs/cgi-bin">
#AddHandler cgi-script .cgi
to AddHandler cgi-script .cgi .pl

Move the folder C:\Program Files\Apache Software Foundation\Apache2.2\htdocs to C:\htdocs and C:\Program Files\Apache Software Foundation\Apache2.2\cgi-bin to C:\htdocs\cgi-bin

Restart/Start Apache (you should have a icon for this in your system tray).
You can test in your browser by opening http://localhost:8080/

Getting Pugs source

The Pugs repo contains the latest spec and tests. So even if you don't plan on building pugs you'll still want to keep an updated copy of the latest source.

Open a command prompt, I'm downloading pugs to c:\temp so I ran:-

cd \
mkdir temp
cd temp
svn co http://svn.pugscode.org/pugs/ pugs

That last command called SVN and tells it to grab the latest version of Pugs and put it in a sub folder called pugs. It'll probably take a while.

From then on you can just update your local copy with
cd \temp\pugs
svn update

Building Pugs

This guide is based on pugs Pugs-6.2.13.14 and Vista32. In the end I didn't actually manage to get it to build. If you do manage it please let me know.

If you are happy with a working Pugs that is a bit older, then you a download a pre-built binary from:-
http://jnthn.net/perl6/
I installed that to c:\temp\pugs_bin

There is a program for Haskell called Cabal. It's the recommended way to build Pugs so we'll have to grab that.
Grab the Cabal source Package from the bottom of:-
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/Cabal
Extract to C:\ghc
Then in a command prompt:-
cd \ghc\Cabal-1.6.0.3\
ghc --make Setup
Setup configure --user --prefix="c:\ghc\cabal"
Setup build
Setup install

We need to do the same for 3 other packages:-
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/HTTP
Extract to C:\ghc
Then in a command prompt:-
cd \ghc\HTTP-4000.0.5\
ghc --make Setup
Setup configure --user --prefix="c:\ghc\http"
Setup build
Setup install

http://hackage.haskell.org/cgi-bin/hackage-scripts/package/zlib
Extract to C:\ghc
Then in a command prompt:-
cd \ghc\zlib-0.5.0.0\
ghc --make Setup
Setup configure --user --prefix="c:\ghc\zlib"
Setup build
Setup install

http://hackage.haskell.org/cgi-bin/hackage-scripts/package/cabal-install
Extract to C:\ghc
Then in a command prompt:-
cd \ghc\cabal-install-0.6.2\
ghc --make Setup
Setup configure --user --prefix="c:\ghc\cabal-install"
Setup build
Setup install

This creates a new executable called cabal.exe in c:\ghc\cabal-install\bin. I'm adding this to my sessions path to make calling it easier:-

set path=%path%;c:\ghc\cabal-install\bin

Now to build pugs run:-

cabal update
cabal install pugs

NOTE: This didn't work for me. Some of the packages failed to install due to regex-pcre-builtin. So I built it manually like the previous packages:-

http://hackage.haskell.org/cgi-bin/hackage-scripts/package/regex-pcre-builtin
Extract to C:\ghc
Then in a command prompt:-
cd \ghc\regex-pcre-builtin-0.94.2.0.7.7\
ghc --make Setup
Setup configure --user --prefix="c:\ghc\pcre"

This failed. After a lot of testing and installing different versions of pcre. I found the line in regex-pcre-builtin.cabal causing the problem was
"Includes: pcre"
If I changed this to
"Includes: pcre/pcre.h"
it would build, but I don't think this is the proper solution...

Setup configure --user --prefix="c:\ghc\pcre"
Setup build
Setup install

cabal install pugs

Still gave me an error on perl5/p5embed.h, i got past this by doing

cabal install pugs --extra-include-dirs=/perl/lib
/CORE --extra-lib-dirs=/perl/lib/CORE

But was then hit by a slough of errors. Much kudos to anyone who gets this to build natively on windows (not under cygwin).

Getting Rakudo source

Open Git Bash, I'm downloading Rakudo to c:\temp so I ran (note slashes are other way around):-

cd /c
mkdir temp
cd temp
git clone git://github.com/rakudo/rakudo.git

(for some reason if you cd / then ls it doesn't show c, but it's there, just try to cd to it)

From then on you can just update your local copy with
cd /c/temp/rakudo
git pull

Building Rakudo

Open a command prompt, navigate to your Rakudo source:-
cd \temp\rakudo
perl Configure.pl --gen-parrot
mingw32-make

This will also download and build the appropriate version of parrot to \temp\rakudo\parrot. It'll probably take a while. Then the mingw32-make will build you a working perl6.exe

Save this as hello.p6
say 'hello it works';

Test it by running
cd \temp\rakudo
perl6 hello.p6

Building Parrot

Parrot gets built as part of the Rakudo install. However this isn't usually the most up to date parrot available, just the one that has been tested to work well with the latest Rakudo. You may want to update your copy of parrot.
However be warned that this may break the Rakudo build.
Open a command prompt:-
cd \temp\rakudo\parrot
svn update
mingw32-make realclean
perl Makefile.PL
mingw32-make

If you want Rakudo to link against this newer version of Parrot, you'll need to rebuild it by:-
cd \temp\rakudo
mingw32-make realclean
perl Configure.pl
mingw32-make

If either of those do fail. Then try doing a realclean before building again:-
mingw32-make realclean
perl Configure.pl
mingw32-make

Getting Perl 5 source

It's worth grabbing the Perl 5 source even if you aren't going to build it. When you get to writing spec tests it can be useful to look at what the Perl 5 tests are doing. Open Git Bash, then:-
cd /c/temp/
git clone git://perl5.git.perl.org/perl.git perl5

Setting up Pugs with Apache through CGI

By default Apache will act like Linux and check the top of each cgi script for the path to it's interpreter. So simply saving your pugs file as .cgi and having the correct shebang should be enough. Although to keep things clearer, I updated my Apache config to treat .pugs files as .cgi also. Do this by editing the line:-
AddHandler cgi-script .cgi .pl
and adding .pugs to the end:-
AddHandler cgi-script .cgi .pl .pugs
You'll need to restart Apache for this change to take effect. A simple Perl 6 CGI hello world script is (your path to pugs might be different):-

_#!c:/temp/pugs_bin/pugs.exe_
print "Content-Type: text/html\r\n\r\n";
print 'hello it works';

Save that as hello.pugs and put in your Apache cgi-bin (such as c:\htdocs\cgi-bin). Then open it in your browser http://localhost:8080/cgi-bin/hello.pugs

Setting up Rakudo with Apache through CGI

By default Apache will act like Linux and check the top of each cgi script for the path to it's interpreter. So simply saving your rakudo file as .cgi and having the correct shebang should be enough. Although to keep things clearer, I updated my Apache config to treat .p6 files as .cgi also. Do this by editing the line:-
AddHandler cgi-script .cgi .pl .pugs
and adding .p6 to the end:-
AddHandler cgi-script .cgi .pl .pugs .p6
You'll need to restart Apache for this change to take effect. A simple Perl 6 CGI hello world script is (your path to Rakudo might be different):-

#!c:/temp/rakudo/perl6.exe
print "Content-Type: text/html\r\n\r\n";
print 'hello it works';

Save that as hello.p6 and put in your Apache cgi-bin (such as c:\htdocs\cgi-bin). Then open it in your browser http://localhost:8080/cgi-bin/hello.p6

Setting up Pugs with IIS through CGI

Open up IIS7 services manager. Then select the website from the left panel (for me it's under COMPNAME->Sites->Default Website). In the center panel there is a link for 'handler mappings', double click. Top right click on 'Add script map' and fill in:-
Request Path: *.pugs
Executable: C:\temp\pugs_bin\pugs.exe "%s" %s
Name: Pugs

When I clicked OK a message popped up asking "Do you want to allow this ISAPI extention?" I clicked Yes.

A simple Perl 6 CGI hello world script is (the shebang doesn't matter on IIS):-

#!pugs
print "Content-Type: text/html\r\n\r\n";
print 'hello it works';

Save that as hello.pugs and put in your inetpub cgi-bin (such as c:\inetpub\wwwroot\cgi-bin). Then open it in your browser http://localhost/cgi-bin/hello.pugs

Setting up Rakudo with IIS through CGI

Open up IIS7 services manager. Then select the website from the left panel (for me it's under COMPNAME->Sites->Default Website). In the center panel there is a link for 'handler mappings', double click.
Top right click on 'Add script map' and fill in:-
Request Path: *.p6
Executable: C:\temp\rakudo\perl6.exe "%s" %s
Name: Rakudo

When I clicked OK a message popped up asking "Do you want to allow this ISAPI extention?" I clicked Yes.

A simple Perl 6 CGI hello world script is (the shebang doesn't matter on IIS):-

#!perl6
print "Content-Type: text/html\r\n\r\n";
print 'hello it works';

Save that as hello.p6 and put in your inetpub cgi-bin (such as c:\inetpub\wwwroot\cgi-bin). Then open it in your browser http://localhost/cgi-bin/hello.p6

Reviewing existing synopses

When we grabbed to Pugs source we got the latest synopses files. If you put them in the same place as me then you'll find them in C:\temp\pugs\docs\Perl6\Spec. Each file is in the POD format you'll be accustomed to from Perl 5. The easiest way to view these files is with perldoc, open a command prompt:-
cd \temp\pugs\docs\Perl6\Spec
perldoc S01-overview.pod

If you prefer viewing things in html you can do
pod2html S01-overview.pod > S01-overview.html
Then open in your browser.

You can also view the files online directly from the Pugs SVN repo.
http://svn.pugscode.org/pugs/docs/Perl6/Spec/

Or online in HTML format at:-
http://perlcabal.org/syn/

Reviewing existing spectests

TODO

Writing Spec for a new feature

You may find (like I did) that a feature you need is currently missing from Perl 6. I asked about it on the #perl6 IRC channel, they suggested that I write the spec for it. When I asked what to do pmichaud (Dr. Patrick Michaud the Perl 6 pumpking) told me:-
Review existing synopses and spectests to see if the feature has already been worked on if yes, then see if it makes sense and point the developers at it.
"point the developers" generally means "file a ticket" if there's nothing existing for the feature, then draft what you think chdir should look like. To draft it, look at other functions in S32 or other places to see how they've been documented.

Also, we don't completely ignore Perl 5 -- so look at Perl 5's documentation and see if you think it makes sense for Perl 6 and just try to adapt it to a Perl 6 sort of idea

The first function I wrote a bit of spec for was chdir. I edited the S32-setting-library\IO.pod file to add in a definition. Now that I'd made changes I needed to use SVN to commit them to the main Pugs repo. First I went onto IRC #perl6 and was given commitbit access (thanks particle). Next I opened a command prompt:-
cd \temp\pugs\docs\Perl6\Spec\S32-setting-library
svn commit -m "added spec for chdir" --username=YOURUSERNAME

This prompted me for a password and then committed my changes.

If I'd added a new file rather than just updating an old one, I would have to of added it to the repo first with
svn add filename
And then done the commit.

Writing Spec tests

Perl 6 tests work much the same way as Perl 5 tests. If you want to add tests for a new feature, or just to fill out what's already there, you'll need to make sure you have the latest Pugs checked out. In a command prompt:-
cd \temp\pugs
svn update

Spec tests can be found in t\Spec\

A very useful tool you can use to see what is already there is ack. If you haven't already installed ack from CPAN you can download it with:-
ppm install ack

From then on you can do fast grep like searches. Such as (looking for spec tests that involve chdir):-
cd \temp\pugs
ack chdir t/spec

Which will return a list of files and lines that contain the phrase 'chdir'. There might also be some older tests written for Pugs that have not yet been updated into the spec, so it's worth also running:-
cd \temp\pugs
ack chdir t/

If nothing comes up you can search on similar features to see what their tests are like. Also if you downloaded the Perl 5 source it's worth seeing how they did it:-
cd \temp\perl5
ack chdir t/

To write tests for chdir, I realized that I'd need to know the current working directory. So the first tests I wrote were actually for $*CWD:-

use v6;

use Test;

plan 2;

# $*CWD is currently just a string

ok( defined($*CWD), 'we have something in our $CWD');

### Get CWD from parrot

my $cwd = Q:PIR {
_ $P0 = new 'OS'_
_ $S0 = $P0.'cwd'()_
_ %r = box $S0_
};
is($*CWD, $cwd, 'matches CWD from parrot');

Test.pm in Perl 6 it like Test::More in Perl 5. I also used Q:PIR to do some inline Parrot. This was saved to t/Spec/S28-named-variables/cwd.t. To upload the new test I opened a command prompt:-

cd \temp\pugs
svn update
svn status

The output from svn status gave me a list of files starting with a ? which means they exist in my local folder, but aren't actually a part of the local svn repo. One of them was t/Spec/S28-named-variables/cwd.t, so I added it to the repo with:-

svn add t/Spec/S28-named-variables/cwd.t

And then uploaded with:-

svn commit -m "added spec test for $*CWD" --username=YOURUSERNAME

Submitting BUG reports for Rakudo

Are you on the latest Rakudo? Rakudo get's updated frequently as it's developed, before submitting your bug make sure it hasn't been fixed in the latest release. See the section Getting Rakudo Source and Building Rakudo for instructions on how to update your copy to the latest release.

If you're sure you've found a bug, then first you need to check if it's already been submitted. You can do this by searching the Perl 6 RT (Request Tracker) at http://rt.perl.org/rt3.

If the bug isn't in there already, you can submit it by emailing rakudobug@perl.org. Make sure your subject begins with [BUG] so that it gets picked up properly by RT and that you send in plain text not html. Your report should include:-
Rakudo commit version
Get this by opening git bash, navigating to
/c/temp/rakudo
and typing
git log
The first commit hash you see in the form
commit long_alpha_number
is the one you want
Parrot revision number
Get this by opening a command prompt, navigating to
cd \temp\rakudo\parrot
and typing
svn info
A description of the bug
Error messages
Your environment (OS, etc)
Example code

Depending on what kind of bug you've found, consider writing a test for it. Tests make the bug easier to fix and help prevent it reocurring in the future.

Writing a new feature for Rakudo

To write code for Rakudo you need to know Parrot (see learning Parrot below). For me, while trying to figure out how to write spec tests for chdir, I found out how to do chdir in Parrot. Therefor I knew most of what was needed to add this feature to Rakudo. After looking through the Rakudo source, and comparing to the synopsis (don't always expect these to match) I came across 'unlink' in src/builtins/io.pir. The synopsis put chdir and unlink in the same boat, so I guessed this was the best place to put it. After checking with the people on IRC the feeling was generally 'if you aren't sure, add the feature... someone else can always move it later down the line'.
I added chdir, and after and few rebuilds and tests I got it working. Now I just needed to get it commited to the main Rakudo github. On IRC Masak was good enough to give me the options:-
"you could submit a patch to rakudobug."
"or you could fork on github, push your patch to your forked repo, and do a pull request."

The easiest way is to email the patch to rakudobug@perl.org with the subject "[PATCH] Description". Git has a couple of options for creating patches. First, make sure you have commited your changes. Open Git Bash and:-

git commit -a -m "Your commit message"

Now you can either run a diff and save it as a patch:-

git diff origin > name.patch

Or use git's format-patch command which maintains your commit message:-

git format-patch origin

Then email the patch and your done :)

To do a fork and pull request visit:- http://github.com/cosmicnet/rakudo/tree/master, login and click on 'fork'. This created a fork in my github account, I opened git bash to create a local branch of it:-
cd /c/temp/rakudo
git remote add cosmicnet git://github.com/cosmicnet/rakudo.git
git checkout b cosmicnet
git pull cosmicnet master
cosmicnet is my github username, for you the above lines will contain your username instead. I now had a local copy of the fork in the branch cosmicnet. As the fork was from the main repo I had to merge in my local changes, and then push them back up to my fork. First I needed to commit the changed files in the master branch:-
git checkout master
git commit -a -m "Your commit message"
Then merge with the new branch:
git checkout cosmicnet
git merge master

Then finally push back to github:-
git push git@github.com:cosmicnet/rakudo.git

From there I was able to do a pull request to the main rakudo repo.

Submitting BUG reports for Parrot

Parrot no longer uses RT to track bugs. Instead they use trac. You'll need to register for an account first, but it very simple. Here is the process I use to submit a bug:-
1) Update to the latest Parrot
cd \temp\rakudo\parrot
svn update
2) Make changes then commit them

https://trac.parrot.org/parrot/newticket

Learning Parrot

docs.parrot.org
mingw32-make html
TODO

Learning Perl 6

TODO

Learning C

TODO

Learning Haskell

TODO

Lyle Hopkins