Now, it's no secret that we are predominately Perl developers. We go to Perl conferences and have fun there. However we also use a lot of other languages, most commonly Scala. In our applications we expect to see Java and Perl happily coexisting. While we're not as active in the non-Perl communities we try to keep informed.

With Jarvis, our asset tracking application, the core feature is it's ability to have completely arbitrary but well-defined and typed documents. It was tough to decide if the foundation model was going to be Scala or Perl based. With previous work we had done in Perl on this subject Perl did come up as the default choice. We were able to leverage the power of Moose and the core Perl 5.x object model. We decided it was easier, and we are opinionated. We do not write Perl that doesn't use Moose and standard tools. It saves us a lot of frustration.

Here's the features that Perl, with Moose, delivered that allowed us to build this thing out successfully:

Mature MOP API, via Class::MOP

Jarvis uses anonymous classes to define document types, compiled and built at startup. This means that every document type has its own class with full introspection and verification. It's ended up with a nearly recursive like structure.

The best data verification utilities we've found (and made)

Moose delivers a type system into Perl that previously wasn't there. We've combined that to create Data::Verifier, which allows constrained type verification with post checks and dependencies. When you combine this with the class introspection discussed above, we get very solid data verification. It also serializes well, so we can store this data.

Well-supported storage APIs

Outside of serialization, we need to store things. The MongoDB Perl API is actively worked on and works very well (thanks 10gen!), and DBD::mysql is rock solid. We're also using DBIx::Class as an ORM to simplify the queries and decorate the RDBMS result sets to have a more similar API to the MongoDB-backed storage entities.

But Perl isn't all there is. For the scheduled tasks and complex event processing we're entrenched in Scala. We're using Quartz for the job scheduling and distributed Scala jobs to inspect the data and emit messaging. The messages are stored and transmitted between the Scala side and the Perl frontend without any issue.

With everything serialized into a common format that is easily processed and parsed we don't have any barriers.

The important thing is making sure that what gets into the system is of the highest quality possible. Ultimately, this is why we insisted on Perl as the language of choice for handling user-input and user-interaction. The strengths of being able to have a flexible object oriented model along with tight type-based extensible data validation made it an easy choice. One we're very happy with as well, as it's proven very effective. The Scala jobs generally just access the data structure in a more raw form, but this is fine because everything is highly testable. It doesn't require the same level of attention as anything user-facing does. This also means that the Scala jobs are simpler and easier to work with.

What isn't good is when that becomes the only tool you are using. All too often developers decide that any given new tool is exactly the only tool they need and proceed accordingly. NoSQL is another example of this.

When we built Jarvis, our new asset manager, we had already built other products to test some various ideas surrounding document-databases and implementation details. The real result is that we figured out that it's really quite trivial and still efficient to split between a traditional RDBMS and a Document database. In Jarvis, we do this by taking all the relational data out and storing it in the RDBMS (MySQL, in this case).

As we touched on briefly in our technical notes, we store the User and Organization relationship information in MySQL. The Organization itself is a document and is stored in Mongo.

The Organization table in MySQL is simply a record that tells us what to look at and has some other denormalized data that we use to keep things a bit faster. This data is highly relational, because there are full permission sets both on the role of a person in the organization and the person's individual permissions (so we have two levels).

This is good because we can traverse the relationships from any permission and see what people directly and roles, then people have that. Doing something like this in a document database is certainly possible but it just doesn't make sense. This is relational data at the core, so we're using a relational storage.

Through a more sensible model class design (instead of just assuming that whatever web framework we use is a good enough API) we have a standalone, useful API that can seamlessly cross between Mongo and MySQL without the application developer having to think much about it.

There are still some idioms based on what storage mechanism is in scope, but by and large it works very well.

This hybrid approach makes a lot of sense and the end result is very nice code, that's easy to follow and extend.  I think we have done a very good job of minimizing any technical debt on a project in its very first stages.

(hat tip to Bradford W. on the title for NoSQL)

We work in topic branches, when ready for testing merge into a qa branch. After QA is complete, it gets merged into master.

We have git hooks in place that every commit to qa then relays and pushes to a series of configured hosts. Each host has a git repository configured to do what it should (update Solr configuration, restart the web application, etc.) This system has worked very well for us, and we just setup per-branch rules on what hosts to push to. This leverages the decentralization that git gives us and the ability to quickly rollback by just moving qa/master to a previous tag.

On that note, it's important to note that we use tags at all the appropriate points in this. We also are evaluating several continuous integration tools which will ultimately be the gatekeeper for what goes out the door.

This will just be an intermediary step where pushing into the master branch will fire off a CI process, and on successful build will release.

We will also have CI on the QA branch, which leads me into...

The Problem

One of our issues is we don't have a long-running development branch and waiting until we get changes into QA for CI to take hold defeats the entire purpose. We're trying to figure out how any topic branch can have CI automatically apply.

The easy and poor solution is having a post-update hook on the branches just set it up and auto-vivify the configuration. We'd like something just a little more formal and easier to track. Still evaluating options, but that's a sticking point we're trying to sort out now.

First I'd like to point out that we proudly detail all the bits we use on Jarvis' About page. We'll do our best to keep that up to date as out technology and infrastructure grow.

Let's start off with a diagram:

nginx starts things off. It talks to Jarvis::Web, our Catalyst application. We use Log4perl to direct logging to a few different log files. Early on we decided to log lots of user activity so that we could analyze it later and learn how our customers are using the site.

We chose a hybrid approach for data. We store user and organization data in MySQL but asset information is in MongoDB. This lets us have a traditional schema for organizations having many users and users that have many organizations. We chose a document database for our asset information because the "schema" we had in mind would be a hot mess in a relational database. The freedom of a Mongo enables some really great features which we'll be unveiling over time.

One of those features is the ability to add sub-types to an asset. You can choose Leased or Purchased and add more data. We've used Moose and some of it's extensions — roles and serialization — to create a powerful system for customizing your assets.

Lastly we come to Solr. It gives us amazing search capabilities that power our datatables in addition to the conventional search. Let's not forget faceting, sorting and filtering!

Jarvis is built from components with a very strong pedigree. We've leveraged dozens of open source projects, multiple programming languages and a vast array of technologies. We look forward to adding more and making Jarvis even more awesome every week!

With YUI3's event delegation and Node API, I think this is the simplest mechanism I've built (and it only looks at elements in the containing form).

Thanks for an awesome JS lib, Yahoo!.

Today you can enter all your servers — along with a lot of data — into Jarvis. Got a lot of them? No problem! You can use our awesome search feature to find whatever you need. There is even a sidebar of search facets to further refine your search to get exactly what you want.

If you lease your servers we have some extra features. You can enter the dates and prices to stay on top of that as well!

I mentioned we had bigger plans. I can't reveal them all yet, but Jarvis will get smarter and smarter. Jarvis will ultimately be able to help you plan for the future rather than reacting. How about a heads up on an expiring lease, a notification of a system update or a hint about a new line of servers?

Sign up for Jarvis today and start entering your servers. Sign up for our newsletter so we can let you know about the new features we'll be adding every week. If you feel something is missing or have some ideas then give us some feedback.

We'll be writing more about Jarvis and pushing out new features as often as we can. Thanks!

The other part is that I saw how quickly the Apple laptops wake up (and sleep). That also made me envious. I've happily trusted Apple to make a device that I want to use to accomplish the task they built the device for. This is an important distinguishing factor, because trying to use an Apple device for a use not intended is very frustrating. This is why I still use Linux servers, and can't imagine ever switching.

With this all in mind, when I looked at the iPad there was a moment of confusion. What is this device? What are the competitors? The UMPC segment isn't there, and the iPad is different than tablets. I honestly don't think there are competitors on the market now (JooJoo doesn't count), but there certainly will be (the HP tablet, etc).

The iPad is the natural evolution of the iPod. The very first generation iPod hit the market with an MSRP of $399. Not very far off of the $499 for the iPad, released 9 years later.

The iPad is pushing the limits of current technology, much like the iPod did back then (for those people questioning this statement, I dare you to look at iPod competitors back then.)

With technology at the point it is now, the current crop of personal media devices is going to be the future. Apple did not do this abruptly, they have been slowly removing and testing the waters. The AppleTV was the first device for media consumption, and it largely failed. The iMac and Mini were full computer that was largely non-servicable by the end-user. The MacBook Air was the closest shot, a closed device without a lot of ports. This was the perfect litmus test to see if people in general were willing to start giving up on things they didn't use.

This is where we are at. We have purpose-built devices that excel at what they're built at doing. In the case of the iPad, it's exactly that: a pad. Much like a pad of paper, this is the new generation. It isn't a computing device, nor should it ever attempt to be one. It will fail at that, just as much as the current "Tablets" fail at being competitors to the iPad.

You don't want background tasks on this. It's a single-task type of device. Push notifications and perhaps event loops are important, but on the iPad (or any other Pad device) you should be focusing on a single task. This is the expectation the device has of its user.

The technology is to be able to quickly flip between tasks. This is where Apple has a leg up on competitors and I am not sure if the competitors are going to truly understand that the quick-task switching is the killer feature. One which Apple does well with, but I still think they could do better.

So after sitting with my iPad for a couple days, both using it as a productivity enhancer and just a media consumption device, I still think they have a hit. it's exactly what it should be. It isn't perfect, but none of the first generation of devices are. I'll be buying the next generation when it comes out, and be very happy.

Pros:

1. Vivid screen. Very impressive, readable and perfect.
2. Orientation Lock is fantastic.
3. It works exactly as you would expect.

Cons

1. It's heavy. I'll build up forearm strength simply reading in bed (a pro?)
2. There is disparity between applications on which way is down. The case should resolve this. Certain applications have fixed orientation (usually games) and that "Down" direction may be counter to what is natural to you, and it doesn't rotate. As an example, Harbor Master's "Down" is contrary to the case.
3. Apps are expensive. More screen real estate does not automatically make something worth more. The app itself needs to be more. There isn't the "14 Day Free Trial" editions on the iPad, and because of that it precludes me from buying a lot of applications I'd like.
4. The App Store. It has too many limitations. No free trials, the iPad/iPhone split isn't clean enough.

Now, each and every one of those cons is temporary and I'm certain will be resolved within the next couple of years. I'll be on my 3rd iPad and probably my 5th iPhone by then, and enjoying every one.

MogileFS has always been my default choice whenever I needed to store a lot of data somewhere, without any relational context to that data.  It handles replication, and for the most part just works.  For things like pastes, with the support for images and other media types, it was a reasonable choice.

Except it is just a bit fragile and lacks certain features that we really wanted to have available to us.  I don't want this to be interpreted as being critical of MogileFS because I think it's good software.  However, I do think that it is software that requires more attention than what we wanted to give it and also is not an exact fit for our purposes.

This discussion was happening while we are also exploring various document-based storage engines (commonly referred to as "NoSQL").  This movement has produced CouchDB and, of course, MongoDB (as well as many others).  For us, MongoDB looked like the obvious solution because it supports replication, has an incredibly easy API and was very fast in our tests.

After about an hour of hacking and debugging last night, I had everything working.  I spent 20 minutes this morning writing a migration script and everything has just simply worked.

With a total investment of about 2 hours — including setup, "apt-get install mongodb" wasn't much trouble — we have replicated MongoDB storage (across 2 Linodes), and new storage code.  I'm pretty impressed with the agility in which we can move to replace a storage backend, and believe it is a testament to the quality inherently available when using superior tools, like Moose, Catalyst and DBIx::Class.

The logo came together very fast.  Hoefler & Frere-Jone's Tungsten is CHC's corporate font so it's use was predetermined.  I had it in my head to create something playful, classy and more pop than my usual work.

From jump I knew that I wanted to set the text on an arc. After a few attempts I finally found what I wanted with a negative leading. This gave it a presence that reminded me of an announcer crowing to the crowd: Codepeek has arrived!

The two-tone effect is intentional, providing some clarification for the words.  The name is inspired by the concept of "peeking at code".  My original idea was "peekcode" but the k and c felt like a linguistic train wreck that refused to roll off the tongue.  Codepeek worked much better but was backward from my original concept.  Placing "peek" front and center in bright white balanced the message I was trying to send.  Careful not to forget the "code" I added a strong gradient and some subtle dots that are intended to look like bubbles rising in the code.  Those embellishments can be easily removed and the logo reduced to two colors very easily, fulfilling one of the rules of logo design.

The small leaves set behind the name are decorative fleurons that provide some distinction for an otherwise wordy logo.  It's a purely subjective decoration that I have an odd affinity for.

Codepeek was a great project that Jay and I took from concept to production in just over a week.  It gave me an opportunity to create a light, playful identity that I'm very proud of.

nolock_cpan[1]> o conf halt_on_failure yes                             
 halt_on_failure [yes]
Catching error: "/etc/perl/CPAN/Config.pm is not writable at /usr/local/share/perl/5.10.0/CPAN/HandleConfig.pm line 286\cJ\cICPAN::HandleConfig::commit('CPAN::HandleConfig') called at /usr/local/share/perl/5.10.0/CPAN/HandleConfig.pm line 213\cJ\cICPAN::HandleConfig::edit('CPAN::HandleConfig', 'halt_on_failure', 'yes') called at /usr/local/share/perl/5.10.0/CPAN/Shell.pm line 393\cJ\cICPAN::Shell::o('CPAN::Shell', 'conf', 'halt_on_failure', 'yes') called at /usr/local/share/perl/5.10.0/CPAN.pm line 375\cJ\cIeval {...} called at /usr/local/share/perl/5.10.0/CPAN.pm line 372\cJ\cICPAN::shell() called at /usr/local/bin/cpan line 198\cJ" at /usr/local/share/perl/5.10.0/CPAN.pm line 391
	CPAN::shell() called at /usr/local/bin/cpan line 198

t/07-debugger.t .......... 
Loading DB routines from perl5db.pl version 1.32
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

main::(t/07-debugger.t:26):	    package Foo;
Unknown error
Compilation failed in require at /home/ems/perl5/lib/perl5/Term/ReadLine/Perl.pm line 63.
 at /home/ems/perl5/lib/perl5/Term/ReadLine/Perl.pm line 63
	Term::ReadLine::Perl::new('Term::ReadLine', 'perldb', 'GLOB(0x8465e28)', 'GLOB(0x83ba9e8)') called at /usr/share/perl/5.10/perl5db.pl line 6073
	DB::setterm called at /usr/share/perl/5.10/perl5db.pl line 2237
	DB::DB called at t/07-debugger.t line 26
Test::Builder::CODE(0x8429fc8)(/home/ems/perl5/lib/perl5/Test/Builder.pm:2426):
2426:	    $Test->_ending if defined $Test;
Attempt to reload Term/ReadLine/readline.pm aborted.
Compilation failed in require at /home/ems/perl5/lib/perl5/Term/ReadLine/Perl.pm line 63.
END failed--call queue aborted at t/07-debugger.t line 63.
 at t/07-debugger.t line 63
Attempt to reload Term/ReadLine/readline.pm aborted.
Compilation failed in require at /home/ems/perl5/lib/perl5/Term/ReadLine/Perl.pm line 63.
END failed--call queue aborted at t/07-debugger.t line 63.
 at t/07-debugger.t line 63
Test::Builder::DESTROY(/home/ems/perl5/lib/perl5/Test/Builder.pm:317):
317:	    my $self = shift;
t/07-debugger.t .......... Dubious, test returned 9 (wstat 2304, 0x900)
No subtests run 
t/08-const-sub.t ......... ok   

Test Summary Report
-------------------
t/07-debugger.t        (Wstat: 2304 Tests: 0 Failed: 0)
  Non-zero exit status: 9
  Parse errors: No plan found in TAP output
Files=9, Tests=73,  0 wallclock secs ( 0.02 usr  0.03 sys +  0.42 cusr  0.07 csys =  0.54 CPU)
Result: FAIL
Failed 1/9 test programs. 0/73 subtests failed.
make: *** [test_dynamic] Error 255
  FLORA/namespace-clean-0.13.tar.gz
  /usr/bin/make test -- NOT OK
//hint// to see the cpan-testers results for installing this module, try:
  reports FLORA/namespace-clean-0.13.tar.gz
Running make install
  make test had returned bad status, won't install without force
Stopping: 'install' failed for 'F/FL/FLORA/namespace-clean-0.13.tar.gz'.
Failed during this command:
 FLORA/namespace-clean-0.13.tar.gz            : make_test NO

 * Fetching {module name}...  ok
 * Unpacking {module name}...  ok
 * Following dependencies...  ok
 * Testing {module name}...   ok

1. Don't inundate the screen with useless messages on success.
   
2. Don't be terse and not include messages on failure.
   
3. Help the user.
   

sudo apt-get install perl mysql-server libdbd-mysql-perl lib-dbd-sqlite3-perl build-essential unzip

sudo su -
cpan local::lib

perl -Mlocal::lib >> .profile
. .profile


cpan Task::Kensho

nothingmuch's blog at http://blog.woobling.org/

It's hard to find someone who knows so many languages and can communicate the knowledge in such a thorough and easy to understand way.  His blog is simply a glimpse into his capabilities, but mostly he provides recipes for simply doing things better.  He has a knack for simplifying common tasks (with the usual side effect of added robustness).

lestrrat (Daisuke Maki) at http://mt.endeworks.jp/d-6/

I like Maki-san for a variety of reasons.  He's probably one of the most entertaining Perl hackers to be around at a conference. He has a really phenomenal view of Perl and programming in general, because he's making a living running a consultancy in Shibuya (Tokyo) and also running the Japanese Perl Association.  When he writes it's interesting to read his perspective, since I think he offers a fairly unique perspective.
The downside of his blog is that many entries are in Japanese.

Stevan Little at http://stevan-little.blogspot.com/

Stevan is the attentive care giver of Moose (founder, guider?) and all around encyclopedia of programming knowledge.  He offers a perspective that is fairly unique, because of his very deep understanding of what seems to be all programming languages.  Oh, and of his crazy involvement in Moose.  It's pretty amazing what he manages to accomplish.

There are many more blogs I read, but lately these have been the most interesting. As my interests or the content shifts, I'll post updates to my favorites (and try to limit to just 3 people in each post).

I don't think she understands why, and I'm not sure if I did either. To be very clear, I'm not a contributor in the truly giving sense of a word. To be fully honest, I contribute in a way that benefits myself. I have learned more from the core hackers than in any job, course or conference. This is my ultimate motivator to stay involved in the community.

The main take away from participating in the Perl community is my own personal and professional development. I learn. I learn quickly and discover many great things I wouldn't discover on my own. There is an additional benefit of finding a way to work on some very fun things.

It saves me time and troubles doing what pays me, so my job gets easier and better as time goes on.

I try to make a difference, but I know that my contributions are nothing remarkable. Anything that I've done that was remarkable was simply because I happened to arrive at a point first.

Even though my contributions are functionally marginal, I feel incredibly gifted and thankful to be accepted in a community as great as the modern Perl hackers.

These inventors of great things spontaneously evolve an idea into a functional product in amazingly short time. To witness these moments is something you cannot get in any office, Internship or academic setting.

I'm involved in Open Source because I've never found a more efficient and thorough vessel of learning. The skills I've developed while simply being around these giants are, at least for me, unattainable anywhere else.

Out of the various Open Source projects I've participated in, none have been as influential as the Perl community. In particular, the Moose and Catalyst sub-communities.

Thank you all who have made this community what it is.

Ruby has a reference implementation called Rack, which takes and shares many ideas of WSGI, but makes it Ruby-ish.

Perl now has Plack, an astounding effort by Tatsuhiko Miyagawa. Plack has been awarded the "Module of the Year" award at London Perl Workshop, and in my opinion is very well deserved.

Within Plack, you can write very lightweight applications just using what Plack offers. You can also use Plack to handle deployment of several other frameworks, including Catalyst (via Catalyst::Engine::PSGI).

As an example of what can be done, I first saw Tatsumaki. Tatsumaki is an event-driven HTTP server that handles streaming (as well as long-poll comet services).

I built an application using Tatsumaki to handle streaming live race results to people who couldn't make it to the track.

It worked out very well, except for a small memory leak (that has been fixed). I had, at peak, 615 concurrent clients with an average of about 380 concurrent clients.

I was using the Plack server directly (Tatsumaki::Server), and having clients connect directly to it. I had a peak throughput of 10Mbps, with an average of 1.6Mbps of data going through Tatsumaki.

I call that a fantastic success, and thank Miyagawa-san for his hard work in getting this software to where it is. Without Tatsumaki, and Plack, I would have been doing a hacky long-poll solution that would have been much slower and less featured.

I also built a YUI3 based Multipart XmlHttpRequest library so that I can stream using YUI3 (I prefer this over jQuery) but IE failed to work, so I had to fall back to long-poll clients which worked more similarly across all browsers.