I normally don’t like making predictions in the beginning of the year any more than I like constructing “top 10” lists. Predictions are hard to make–I can’t even accurately predict what I’ll be doing six months from now, let alone how millions of people I don’t know will behave.
But like making top 10 lists, making predictions, especially for the new year, is seductively alluring. I wasn’t going to succumb to it, but here I am. But instead of making predictions per se, I’d like to talk more about what I expect to be doing this year, which as an experienced developer, might be somewhat interesting. Then, I’d simply like to speculate on how things that are already happening might play out.
Erlang, Fedora
In my day job, I work with digital library data that for any given collection can be in the multi-terabyte range. As you can imagine, this is challenging in terms of processing, storage and network transfer. As a result, I have been very interested in Erlang recently, and am starting to make my way through the Programming Erlang book by Joe Armstrong. At least in terms of processing large amounts of data, the massive scalability of Erlang looks very appealing.
I’m not so sure what the larger, long term impact of Erlang will be. It seems to be a niche language, although the niches it serves are significant, and it seems, growing. Also, Erlang has been around quite some time, which cuts both ways–its robust and proven, but also feels like an old language as many have pointed out. Either way, at some point this year, you may start seeing Erlang code in this blog. Also, as I have a major existing investment in Java, I am interested in seeing how Erlang and Java might coexist in the same environment. Can I leverage existing Java code, but still take advantage of Erlang’s scalability?
Another tool I am investing heavily in this year is Fedora. That’s the digital repository software, not the Linux distribution. Fedora has been around for a number of years now, more than a decade if you include the initial research and its immediate predecessors that gave rise to it. In the last year, however, it has entered a new phase of development and started to get attention in the larger open source community outside of libraries and museums where it has been traditionally used.
Fedora grew out of semantic Web research, and I tend to view the semantic Web the way I have always viewed EJBs, it seems like an overgrown academic exercise that will never achieve its stated goals. I think that as the Fedora community has grown, it has become more practical, however, and in my case, I see it as a standardized, generic storage layer on which applications can be built that reuse the digital content of the repository. In particular, I am interested in connecting Fedora with new delivery mechanisms and ecommerce applications, so I am looking to use it in conjunction with JRuby on Rails and possibly Magento (more on that below).
The recent growth of Fedora is happening at an interesting time. In 2007, I saw a lot of musings on various technical blogs about the perceived shortcomings of relational databases, and whether or not traditional RDBMS’s are even appropriate in the Web 2.0 age, which resulted in all the controversy you would expect. I think something may have been missed in those conversations, however, which is that there are times when you need to manage information that is unstructured, arbitrary, or only known at runtime. In other words, you can’t fit it into a relational schema–even a very generic schema–as you are designing the application. You need flexibility that the relational database doesn’t provide, and in fact, these situations arise more often then we probably realize because we’ve gotten used to working around them, such as storing data as XML alongside relational data.
Fedora may offer an alternative, at least for some of these situations, but others also exist. The most intriguing is CouchDB, written in Erlang. I can’t give a good compare/contrast analysis of Fedora and CouchDB, and at this point, I am not even sure how often they might overlap to address the same application needs. But assuming that they might overlap, I would make the initial superficial observations that CouchDB seems to be directly and practically addressing the need I mentioned above, and as Fedora is a straight Java application with some semantic Web baggage, it looks decidedly middle aged compared to the sexy young CouchDB, which began not as an academic exercise, but as an accessible tool to address existing needs.
However, Fedora has legitimacy in the library world, and a significant track record of success, so it is a good fit for a few projects now in my pipeline. But, I will definitely be keeping my eye on CouchDB, especially given my interest in Erlang, and in the future, I may be able to talk a little more intelligently about it.
Finally, it seems that Java is once again entering a crisis, as many seem to be coming to the same conclusions as Bruce Eckel, and I admit, I find it hard to disagree with his analysis. Its also interesting to see people suggesting that Scala should be adopted as the “new Java”, although I think that’s a little premature.
I long ago started seeing Java less as a language and more as a platform, and the rise in popularity of JRuby and Scala are squarely in line with this thinking. As a result, I’ve become less concerned about the evolution of language itself, at least in terms of the code I write on a daily basis. Either way you look at it, Java is now a legacy language, even if its still widely and actively used for new development, its use today and in the foreseeable future is based on past success. I think we need to start looking at its evolution and support from that perspective.
PHP, Drupal, Magento
I’ve been thinking for some time of writing about PHP and why I often find myself using it in certain situations, despite the fact that I am quite familiar with its shortcomings, and I have expertise in other environments. If you’ve read this blog before you know my philosophy is that technologies are tools. No tool is perfect in itself and most likely no tool is the perfect fit for any given task. The best developers know many tools, so they can determine the best tool for the job, based on knowledge backed by experience. Choices generally come down to balancing trade-offs.
As a result, I do cringe at those who blindly evangelize Ruby on Rails, just as I cringe at people who insist on apologizing for Java and those who completely dismiss PHP. Usually these are young developers, or developers who have never bothered to learn anything new, and they are fearfully clinging to the one technology they know. If you are only comfortable with one technology, it doesn’t really matter what that technology is, you can’t claim to be a knowledgeable developer.
I have been interested in the recent exchanges about Dreamhost’s hand wringing over Ruby on Rails, the most insightful of which was Ian Bicking’s analysis. I can’t add much to that analysis, except to say that the importance of PHP’s ubiquitous availability, not only on commodity hosts, but alongside most Apache installations, can’t be underestimated. If you are creating software, whether its for monetary compensation, or the good of humankind, this ease of deployment is crucial. You may not like programming in PHP, but better you take the pain of developing in a language you don’t like rather than subsidize development in your favorite language with a more complicated deployment for your customer. The truth is, you’ll probably have to deal with those deployment issues on their behalf anyway. Better you should absorb the pain yourself, in an environment that you can control and manage and make things easier for your customer, and for yourself in the long run. Especially if you are selling software, either directly or indirectly by charging for your time, its important that you make it easy for the customer to buy what you are trying to sell. (Think about how 37signals offers Basecamp, a strategy that boils down to controlling the deployment environment.)
Now before those one tool Ruby folks jump out of the bean bag chairs in their mothers’ basements to shout me down, let me repeat that I am not saying that PHP is better than Ruby or vice versa. I am just recognizing what I think is one very important reason for PHP’s popularity, and I think that if Ruby based Web deployment was simplified, it could open new possibilities for those applications. And obviously, I am not suggesting that the deployment model mirror PHP’s; I think that JRuby provides an important step toward simplifying Ruby on Rails deployments for example, by allowing you to deploy a WAR file in a standard servlet container. Suffice it to say that I think this is an area of improvement for Ruby Web frameworks, and with the rise in popularity of Rails, Merb and others, the time is right to prioritize this area of development.
It struck me recently how PHP has become the de facto development environment in the minds of many business people I meet, something I never would have guessed would happen, say six years ago. But it seems that especially in mid-size businesses, which I tend to serve the most, PHP is viewed as robust and well supported, with wide availability of expertise (and yes, perhaps inexpensive expertise).
All these reasons are why in my freelance work, I find myself coding overwhelmingly in PHP. Its often presented by clients as the target environment, or, I may be asked to provide a service that is well understood, such as content management, blogging, wikis, ecommerce solutions, etc., and in each of these categories, there is at least one open source, best of breed option available in PHP, and often several that can be installed and customized easily for the customer. Again, PHP is a good fit, its a logical choice from several angles.
Recently, I have been getting more deeply involved in deploying Drupal based sites, and I am only now starting to understand the true power of this software. I originally just saw Drupal as a way to easily offer basic CMS, blogging or wiki functionality in a flexible manner, but now I am starting to think of Drupal as a platform on which to build many different types of sites, including custom applications that can be built rapidly using Drupal’s core API.
I recently purchased and read Pro Drupal Development by John K. VanDyk and Matt Westgate, which is outstanding, and should be considered as THE MANUAL for Drupal. As I read it, I gained a tremendous respect and appreciation for how well architected Drupal is. It has a very strong, generic foundation on which everything is built, and even as functionality is heaped on that foundation, it remains organized and flexible. Anyone who thinks that you can’t develop a well designed, robust application in PHP should read this book and look at Drupal.
The book itself is a must have for any serious Drupal developer. Although Drupal has a great support community and online documentation, this book really exposes the inner workings of Drupal, explains coding best practices and gives a comprehensive look at all areas of Drupal including security and scaling. I was also surprised by some of the things I learned from the book, such as Drupal’s installation profiles and the fact that JQuery is part of the core. Despite being part of the Drupal community for some time now, I didn’t know these pretty fundamental features.
I have also been impressed by both the number and quality of the contributed modules to Drupal, and thanks to the underlying Drupal API, how well these modules tend to work together. Some of these are quite large and complex, such as CCK (for creating custom content types with minimal hand coding) and the ecommerce module, and some are relatively small, but offer important functionality in key areas. Often, a feature that you want will be served by several alternative modules, but unlike other module collections I have seen for other frameworks, these alternative Drupal modules are often all quite mature, and provide you with flexible alternatives for your needs.
To give one interesting example, there is the OpenResort module, which builds on the ecommerce and other modules to provide a full featured reservation system for “accommodators” which in the case of a site I created recently, meant a Bed and Breakfast. This module allows the owners of the B&B to describe the features of each room and their availability and allows customers to make reservations and online payments.
The ecommerce system itself is quite flexible, allowing for the purchasing of tangible products as well as things like file downloads and even subscriptions to your site, if you offer “premium” content. And of course, Drupal makes it pretty simple to create custom modules that integrate with the Drupal core and play nicely with other modules by following certain conventions.
And on the PHP/ecommerce front, I would like to put in a plug for a very interesting project that should see its 1.0 release this year, Magento. Although I don’t think Magento is that widely publicized yet, its positioning itself as the next big PHP based ecommerce platform, and from what I have seen, I think they may just succeed. PHP still seems to be dominated by osCommerce and its forks, which have been showing their age for some time now. There are other good open source PHP ecommerce solutions, but in my admittedly superficial investigations, they don’t offer a truly compelling alternative to a solid osCommerce fork like Zen Cart for example. Magento could come to be the best overall PHP ecommerce offering as osCommerce once was.
Rails, Web 2.0, Web 3.0
I think that this will be a transformative year for Rails and social networking sites. Lets start with Rails. I tend to see Rails as an approach, a concrete philosophy and set of best practices for Web application development, for which Ruby on Rails is both the reference implementation and the sandbox in which this approach evolves.
For example, I recently consulted on a project in which I recommended using Ruby on Rails, but because one of the decision makers was a former Perl programmer, he wanted to use Perl(!) for new development–and the decision was made against my advice. I was initially quite disappointed by this decision (and I say that as a former Perl programmer), but then I worked with the main programmer on the project who found Catalyst, which appears to be a very faithful Rails clone for Perl. After auditing the code for the initial prototype, the idea of viewing Rails as an approach really hit me. If I needed to, I could step into this application and from my RoR experience, know my way around, and be well prepared to debug it, for example. As there is at least one Rails clone in just about any mainstream Web development environment you can think of, a programmer can code in a language they knew well, but use an approach that any Rails developer can understand and support (assuming that Rails developer is not one of those one tool developers that only knows Ruby).
One thing that has always impressed me about Ruby on Rails is not only how well it codifies certain practices, but also how forward looking the framework is. Ruby on Rails not only represents what we have learned about Web development in the past, but it is actively influencing the continued development of that knowledge–perhaps most obviously with its evolving RESTful features–and that more than anything, is why I think the Rails community will continue to grow in the foreseeable future.
My hope is that as JRuby continues to mature, the Rails approach will be recognized as just that, an approach, a philosophy, or however you want to characterize it. I think that’s an important, if subtle, distinction that needs to be made so that the approach can continue to evolve. I also think that its a crucial step to take toward figuring out what approach comes after Rails, and may even help us figure out what comes after Web 2.0.
And speaking of Web 2.0, I think this year we will move beyond collectively taking for granted the basic functionality of social networking sites like Digg, Flickr, Twitter etc., and hopefully move to the next phase of creating truly interesting applications that evolve existing Web 2.0 concepts, which might begin with making these services somewhat interoperable, possibly utilizing OpenIDs. I agree with those that think that Web 2.0 should really be thought of as Web 1.0, because it seems we are just at the very beginning of understanding how to develop resources that naturally fit the Web environment. Last year everyone was wondering what Web 3.0 would look like, and I think this year we may start figuring that out.
The overlapping rise of Rails and Web 2.0 is historically interesting itself. Just as I think we are only starting to figure out what kinds of resources and services to provide on the Web that play to the strengths of that unique landscape, I also think that with the Rails approach to development, we are just starting to figure out how to design and code those resources as well. Especially with Rails’ aggressive adoption of AJAX and REST, we may see the two areas develop in close parallel.