Continuing the writeup of the recent NorthWestEngland.pm hackday (Part 1):

Server

We’d originally assumed that the main backend would be EC2, purely because it seems to have more mindshare currently (for example, the opensource initiatives around Eucalyptus and OpenStack seem to be targeting the EC2 API as a de facto standard). However Marco Fontani (who has recently set up Glasgow perlmongers, and herded a group of 3 all the way down to Lancaster, yay!) has already published Net::RackSpace::CloudServers to CPAN and was keen on targeting this. He and Iain worked on the Rackspace backend for Oyster, while Carl (fade) and Ian Norton (idn) looked at the EC2 backend. The 2 backends work rather differently from each other: while in the case of EC2, we just have to inflate a stored image (AMI) that’s stored on Amazon’s S3 storage service, for Rackspace, it seems that we have a limited choice of prepared images, and then have to build a server from that.

The commonality of course is a routine that, from a fairly standard (Debian or similar) image, creates a base Oyster image. This will either:

• be stored as an AMI for inflation into EC2
• OR be run when a machine is provisioned into Rackspace.

The Rackspace backend is slightly more advanced, though both are now capable (sometimes with a bit of manual kicking required) of getting launched from the client. Next step is to get them exposing the hooks required by the Deployment team!

State, and configuration

One of the interesting things about splitting up into the 3 teams is how (at best) this could help us maintain encapsulation. We had a bit of discussion where the Server team would have liked us in the Client team to pass on the information from the command line to them, let them write information back to our config file, or require us to maintain state. Though they make some good and interesting points, I think it’s a good idea to at least be wary of confusing the concerns of these 2 areas, and I think we’ve made the right decision to defer any strong coupling between the areas at least for now. (For example, configuration files might be entirely irrelevant, in the case where the server is being launched by a script, or via the planned managed Oyster web-service.)

Deployment

Miles (pozorvlak, another of the Glesga boys), Graeme (grim), and Paul (bluefoo) worked on this, and specified that we will, for now, do deployment via push to git. This is also the technique that Heroku, a massive influence on Oyster, appear to do use. While this is great for an entirely managed service, there was some discussion about whether it was the right approach for us – for example, if someone chooses to plug in a managed git service, such as github, there may be limits on how we can incorporate git hooks. But as the team have real interest and experience with git, it makes loads of sense to do it like this for now, and we can always look at extracting the relevant code into (for example) Subversion hooks, or a standalone script at a later date! Matt also suggested that the deployment should honour the Perl de facto standard of a tarball as the “unit of deployment”.

I think we were all unclear about how deployment would work, which led to some frustration – discussion/post-mortems welcome!

Marketing, testing, documentation

Lateef, also from Glasgow, and Gabi (gabimuc) who had made it all the way from Germany, worked on testing and documentation. I didn’t pay nearly enough attention to what they were doing (bad osfameron!), and I do think I personally squandered them as a valuable resource for the day. (Possibly falling into the “but it’s too early to test/document things” trap) I’d love to get a writeup of experience from them to learn how better to incorporate their skillset into future hackdays.

That said, their help was already incredibly helpful!: several times, gabimuc tried to follow our documented API and pasted error traces, which helped us debug our assumptions and documentation. I think it might have been even more useful to have her and Lateef moving around the teams, testing and discussing assumptions and design throughout? Discussion welcome on how to improve this for next time!

Mark also did some nice work on thinking up an Oyster logo, currently work in progress. The marketing of the project will really kick in at some point, and I’m sure we’d appreciate new contributions for artwork/copywriting too.

Next steps

The next main step is to get, by hook or by crook, a “hello world” type app running on one of the backends (this could be a Catalyst app, or a simple Plack/Mojolicious one), the benefit of which would be in a) proving that the components work together, and b) to get the boost of “look, something worked!” which we just missed at the hackday itself ;-)

I’ll be presenting a talk summarising Oyster and its current status at London Perl Workshop 2010 this coming Saturday. Hope to see you there!