Cold Hard Code

April 2009 Archives

Speaking at Japan Perl Association Seminar #2.

By J. Shirley on April 29, 2009 10:23 PM |
Comments welcome

I also spoke at the Japan Perl Association Seminar #1, at Akihabara.  I wasn't easily able to blog about it at the time, and I'm giving the same talks for Seminar #2 so no big deal.


If you're in the Kyoto/Osaka/Kobe area on May 14th, please stop by!

The slides, both in embedded format and a PDF with detailed notes, are below. 

Better Perl Practices



Better Perl Practices (PDF with detailed notes)

Catalyst Tips: 10 things you should know



10 Catalyst Tips (PDF with detailed notes)

Comments welcome

Better Testing.

By J. Shirley on April 27, 2009 7:19 PM |
Comments welcome

With all the thought I put into this title, I really have only those two words to say.  I personally hate testing.  I hate singular test files even more.  I find that I have to copy and paste a lot of code.  The next step is to shove some stuff into upstream packages that abstract out tests.  This just makes things uglier, and in my opinion, harder to test.

I was very happy when I saw Test::FITesque hit the CPAN.  The reason is that it separates out what you are testing versus what you are testing with (in terms of data).  You also can establish different routes of tests, and between the magic with Test::Builder, it all Just Works™.

Even though I read the documentation and witnessed a talk by Scott McWhirter (konobi in the perl world) I was still not certain of how it all came together.  My first attempt was hideous.  After several increasingly positive tests, I now cobble together a test suite that looks like this:


package Roostermatic::Test::Team;

use parent 'Test::FITesque::Fixture';

use Moose;

with
    'Roostermatic::Test::Fixture::Schema',
    'Roostermatic::Test::Fixture::Team',
;

1;

As an aside, the samples here are coming from the soon-to-be-on Github Roostermatic application
.

So, as you can see I've created Roles that have each individual test.  While the container is a Test::FITesque feature, I could use Test::Builder to create a test runner outside of Test::FITesque.

This, in conjunction with Test::FITesque allows me to have another layer of test assembly and abstraction.  My actual script for running the tests looks like:


use Test::FITesque::Test;

my $test = Test::FITesque::Test->new({
    data => [
        [ 'Roostermatic::Test::Team' ],
        [ 'init_schema' ],
        [ 'team_create', 'Liverpool' ]
    ]
});

$test->run_tests;

The individual line items after 'Roostermatic::Test::Team' (and a side note on the naming, in retrospect I like Test::AppName better than AppName::Test) instruct Test::FITesque to run those tests.  Those individual tests are defined in each role.

Here is my Schema role:


package Roostermatic::Test::Fixture::Schema;

use parent 'Test::FITesque::Fixture';

use Test::More;

use Moose::Role;

use Carp;
use DBI;

has 'schema_class' => (
    'is'    => 'rw',
    'isa'   => 'Str',
    lazy    => 1,
    default => 'Roostermatic::Schema'
);

has 'connect_info' => (
    'is'  => 'rw',
    'isa' => 'ArrayRef',
    lazy  => 1,
    'default' => sub {
        [
            'dbi:SQLite:t/var/test.db',
            '', '',
            { quote_char => '`', name_sep => '.' }
        ]
    }
);

has '_schema' => (
    'is'        => 'rw',
    'isa'       => 'DBIx::Class::Schema',
    lazy_build  => 1,
);

# Define the lazy builder here to actually connect to the schema as configured
sub _build__schema {
    my ( $self ) = @_;
    my @parts = DBI->parse_dsn( $self->connect_info->[0] );
    if ( $parts[1] =~ /sqlite/i ) {
        if ( -f $parts[4] ) {
            unlink($parts[4])
                or croak "Couldn't unlink $parts[4], deploy may fail";
        }
    }

    Class::MOP::load_class( $self->schema_class );
    my $class = $self->schema_class;
    my $schema = $class->connect(@{ $self->connect_info });
    # die if we can't connect?

    return $schema;
}

=head1 METHODS

=head2 schema

Read-only accessor, returns a connected schema

=cut

sub schema { shift->_schema; }

=head2 init_schema

Runs 2 tests, and returns a deployed and initialized schema

=cut

sub init_schema : Test : Plan(2) {
    my ( $self, $params ) = @_;

    my $schema = $self->_schema;
    ok($schema, 'schema connect');

    eval { $schema->deploy };
    ok(!$@, 'deployed schema');

    return $schema;
}

1;


Pretty simple, I think.  I can expand on the tests and mix the roles to create more advanced and complex tests and still use the user-friendly nature of Test::FITesque to create higher level suites.

On a continuation of my previous mention of naming, going forward I'm refactoring into a different naming scheme that I think is more semantic.

Project
A project is comprised of many high level goals. This is the configured test run, as verification a project is done
Goal
A runnable and complete test group is called a Goal (this would be a Suite in most nomenclatures). This is just to test a specific aspect, a line item in the project
Achievement
A goal is made up of several achievements.  This is a higher level than a unit test, and is what I am using as a role above.
Unit
An atomic unit in an achievement is a unit test (nothing different there).
I'm honestly not sure if any of this has been said before, and I'm probably negligent in my readings and keeping current with this.  If anybody knows of good readings, please let me know.

Comments welcome

Remember: MVC for the Web, may not be MVC™.

By J. Shirley on April 24, 2009 4:55 PM |
Comments welcome

This is a precursor to Catalyst Input Validation with DBIx::Class Models

I just recently gave two talks at the first Japan Perl Association, and one of the things I didn't focus on was design of your Catalyst application.  I opted to not talk about this, because I feel it to be a subject beaten to death.  In hindsight, I guess it isn't as a lot of people don't take the initiative to search through mailing list archives, and there needs to be some more public discussion on the matter (outside of mailing lists).

First, please try to forget the concept of MVC if you aren't already familiar with it.  Without a lot of thought, you're possibly going to go down the wrong path.  Instead, you need to think about what things are actually doing.  In a way, this is a role (not in the Moose sense).

The way I see it MVC is better extrapolated as:

  • User Facing
    • User Input
    • User Output
      • Visible Output (Template Rendering)
      • Invisible/Async Output (Emails, Long running jobs)
  • Business Facing
    • Business API Layer
      • Backend Implementation
Each indentation level is significant of separation of pushing things further down.  As you can see, in my perspective you don't really have the MVC layouts in the same way, even though it really is.  User Output is the View, both asynchronous and the user-facing (html) view.  The User Input is the Controller.  Together, they are combined to create all the user facing aspects of your system.  This is, unfortnately, why so much business logic gets placed there.

There is no longer any compelling reason to use the Business side of things.  It seems a superfluous add-on without an added benefit.  The reason for it isn't obvious for the simple cases, and this is one fo the reasons why I think other frameworks are met with more success.  They don't use the nomenclature of a non-intuitive system, but that's just naming.

The primary reason for having your business aspect is that it shouldn't be specific to any application.

This is all great, but in practice sometimes this gets very difficult to mentally categorize.  For example, to create a user account, it is tied to an application.  Sort of.  There are application specific rules for how an account is created, and validated.  In many instances, you have to submit an email address, and then click on a verification link.

If the backend tools allow for creation of a user, it is a different path; possibly, this case is even a different application.

This is why it is important to put your business rules on what constitutes a valid user in your backend business model, to take with you between applications and use cases.  Your transitory state on these objects (an unverified user) can be tracked in the controller, assuming your business layer has a simple API into the interface.

This means that code for upgrading a user to a verified user should be as simple as:
$user->verify_and_upgrade( $code );

If that throws an exception, such as a bad code, then the application needs to handle that.  This is the controllers job.  This means that the controller just applies its application specific rules, but all the validation and actual upgrading of the object happens behind the scenes, exposed by the very friendly "verify_and_upgrade" method attached to your user object.

To create the degraded user, there should be a different method for that.  Rather than code in your controller creating a user and setting some validation flag to false, there should be a simple API method (in the Catalyst & DBIC world, this would be):
my $user = $c->model('Schema::User')->create_unverified( $data );
my $code_to_email = $user->verification_code;
As you can see, this also keeps the controller code very simple and easy to follow.

The main principle here, which I think is paramount to delivering quality applications is simply:

Focus on the API

If you focus on your business layer having a very good API to do all functions, no matter how trivial, your controller and application specific has no choice but to be thinly written.

Rather than try to adhere to good programming practices through will power, focus on just building an API first.  Then all that other stuff just happens, for free.

Comments welcome

Varnish and MovableType for content management and application integration.

By J. Shirley on April 14, 2009 10:18 PM |
Comments welcome

I encountered an interesting problem, without any clear solution in front of me.  To save you from a lot of reading, I just deleted a 3 paragraph rant about the software I'm replacing.

Now, what I have is a 3 part system.  First is the Catalyst application that runs in a secure environment and handles purchases and registrations.  The second larger component is the content management, which is comprised of the legacy system and the new Movable Type based system.

MovableType was a fantastic choice, for the static publishing and easy user interface.  While the asset management could be better, I can always fix that for our needs after the fact (which I will be, most likely).

The problem is getting 3 applications to live together harmonously when they're very disjointed.  I don't really want to server the MovableType CGIs, so I can block any .cgi scripts at the frontend.

MovableType would be able to handle everything together, except for one caveat: A user logged in, with a valid membership, is presented with different content.

This, quite obviously, is a fantastic case for ESI.  I think most people in the MovableType world would have their templates omit PHP and put the generated HTML files behind Apache and PHP. 

Hell no.

There's 3 reasons for this:

  1. I hate PHP
  2. I don't want the bloat of dynamic pages when it isn't necessary
  3. I already have an application that has all the snippets of templates, and uses a unified CSS structure
So, the only thing I'm missing is a controller that renders the atomic units of the logged in snippets. 

This primarily consists of a few member benefit pages (showing redemption instructions) and menus, as is the typical case with ESI.

Now, this doesn't make sense in just words.  So here's a picture that also doesn't make sense:

Using ESI in front of MovableType
In a nutshell, all that is really happening is a server-side include.  It's similar to PHP, except you have a proxy-based load balanced backend cluster that can be accessed via an internal network (or on the same server, if you're a single-server setup).  The ESI processing can be very simple to setup, and all your application needs to do is spit out the little snippet of a page.

The downside of this is your application processes one request for every ESI include, which can honestly get a bit costly but there are some better perks here over letting a browser do it.  First, Varnish can use a keep-alive socket and stream the requests better, as well as providing caching on the individual ESI components and store the individual components inside of its main cache.


Comments welcome

Continue reading Varnish and MovableType for content management and application integration.

Better applications with MooseX::SimpleConfig.

By J. Shirley on April 14, 2009 3:54 AM |
Comments welcome

First, a short preamble to the recipe I will present.  I had been trying hard to write up an entry here once every two weeks, focusing on substance.  I was falling behind on a draft entry so it was looking like 3 weeks was going to lapse before it was released.  It's a good article, though, on using Git and FastCGI for deployment management.  It was requiring some code-as-I-go, as it was largely a play thing and I'm not using it in production (which is something I typically refrain from writing about but in this case I was making an exception).  I was doing well, and tragedy hit and I lost my awesome nephew.  Since then, it's been very hard to engage myself in anything of importance, and I've really struggled.  Now, nearly 2 months after the terrible event I'm mentally recovering.  The outpour of support and encouragement has been amazing, and I'm truly grateful.  Now, on to business with a completely different recipe...

I posit the following statement which I hold true: Command-line applications are typically called with the same options specified.

With this in mind, it seems a gross oversight that the most commonly used command-line tools don't support configuration files to populate options and default state.  I would personally love to simply be able to do mysql --configfile work.yml.  Unfortunately, I am unable to.

Fortunately, Perl and Moose lovers can.  Allow me to introduce you to MooseX::SimpleConfig.  This fantastic role gives you the capability to bake in configuration into your application with very little fuss.  If you use MooseX::Getopt, most things are effectively free -- including the --configfile command line option to explicitly specify the configuration file.

Things, however, get better from there.  I found myself wanting to setup local overrides of configuration.  I desired something similar to the way that Catalyst merges in from myapp_local.yml, to myapp.yml (or .conf) and finally into the Perl packages for defaults.  What I ended up with is a way to just traverse a list of paths, and load the first one found.

It's a dead simple recipe:


with 'MooseX::SimpleConfig';
with 'MooseX::Getopt';

has +configfile => (
    default => ( grep { defined $_ and -f $_ } @list_of_places_to_look )[0] || ""
);

Now, you just need to setup your @list_of_places_to_look array, and you're set.  The first one found wins, and that's the config file.

To take it a step forward, on an application that has different projects or some othe binding feature, you could shorthand this using environment variables to bootstrap the project for smaller and more meaningful command lines:


# Populate like this:
my @list_of_places_to_look = (
   File::Spec->catdir($ENV{PROJECT}, 'etc'),
   File::Spec->catdir($ENV{HOME}, 'etc')
);
# And then:
$ PROJECT=foo myscript.pl
# Profit!
Now, if all you are doing is using your environment variable to set configuration paths, I'd say you are going down a wrong path.  However, using the project dir in a more substantial way (such as a Cwd call and using that project path as a place to store sockets or PID files) then I think it's a safe usage.

You'll see more of this when I finally do finish up the FastCGI and Git recipe, so stay tuned.

Comments welcome