September 2006 Archives

I previously published an article here about using Drupal and RT together to create a knowledge base system. I've continued to improve the system and wanted to give an update on how it's working.

Of the pieces I mentioned in my previous article, I have implemented Organic Groups in the system. I have not (yet) implemented any of the other ideas I had at the time. In addition, I have used the Front-Page module, added use of the Content Construction Kit (CCK), and started adding some tweaks to RT.

Organic Groups

I have added Organic Groups to the site to operate the same kind of functionality as is provided by a SharePoint sub-site or a Trac project. Each Organic Group is a smaller segment of the whole knowledge base dedicated to a specific project or knowledge area. Organic Groups allows each user of the site to be a member of just the groups that are of use to him. It keeps membership lists for each group and also provides (some) notification regarding new content (it would be nice if content updates could be included as well, but does not at this time). Content may also be associated with no group or can be associated with a group while also being public as well.

All in all, this has been very helpful to segmenting the site.

Front-Page

I've used the front page module to setup a simple comment that goes to all anonymous users. The knowledge base is available from the Internet, but is only intended for internal use at this time. However, it is possible that someone might stumble upon it (most reading this could probably guess it's location in less than three tries and probably a majority only in one...). I wanted a special page facing out for anonymous visitors so that it would tell them to bugger off (though, in much nicer terms than that). It would also help our staff find the pages they are looking for more easily.

The front page module allows me to setup a front-page that's special to any anonymous visitor as well as having a page that's special to those who have already logged in. After you login to the site, the new front-page is a basic table of contents and list of policy documents that affect all of the group sites.

Content Construction Kit

The CCK is the future of all content on Drupal sites, as far as I can tell. Soon, the "story" and "page" and "blog" content types will just be custom content types created by the CCK. This is a good move. Custom content types has been a weakness of Drupal for sometime and flexinode wasn't quite there—in fact, it was one of those demo-turned-indispensible modules that every programmer dreads creating or maintaining.

In this case, I've used a combination of the CCK, the Category module, and the Event module to create a special content type, the "sub-project". These are a combination of "component" and "milestone," which seems to fit our web development project. Some of the features on the site can be grouped into convenient sets that all can be accomplished together. Each of these tasks are related and often feed each other, but aren't really the same task, thus are handled in separate tickets. This has been very helpful.

Each sub-project has a title, a description, and a due date (the need for the event module). These are plottable on a calendar via the event module. These are also "categories" via the Category module, i.e., nodes that act like taxonomy terms. This allows knowledge base content to be classified into these special categories. This is very cool. I have also pulled these into RT, which I will cover in the next section.

In addition to the above, I plan on creating some other custom content types for various bits. For example, I would like to build a "computer" content type to describe individual machines on our network and a "software package" content type to describe computers. These can then be used in the IT group to associated particular articles with particular computers and software packages. Thus, the CCK content types begin to take up the role provided by custom lists in SharePoint. These can also figure into the RT modifications I'm about to describe.

Auto-Complete in RT

I need to formalize this a bit better, but I have added customizations to the custom field rendering component in RT so that I can make custom fields autocomplete in the same way as they do in Drupal. I have added a "Sub-project" custom field to the Web Development queue. Then, I tweaked the component so that those custom field editors pull the available values from the available milestones in Drupal. This is the first real communication I've added from one system to the other and, so far, to work it requires you to be fully logged in to both systems (no single sign-on yet).

I'm still pretty happy about this and it certainly beats the pants of Microsoft SharePoint. I hope to have further updates as well. Currently, I'm thinking of more CCK types (as mentioned), pulling tickets into Drupal from RT so that categories list the tickets that belong to them (possibly using RT's RSS features and the Aggregator module or Aggregator2 module or some other variant), and getting single sign-on working so that we don't have to login twice anymore.

I'm also still considering some of the things I mentioned previously, though they aren't quite as high on my radar right now.

Cheers.

Well, I've been casting about for a topic to blog about on this day off and I've finally got one that I can get traction with, I think. If you're reading this, then I must have published it. :)

I've been doing a lot of work building Drupal sites of late and I've decided that I really like Drupal for a number of reasons, but I really despise it for others. This is a common theme on my blog: I like things, but there are a lot of things I don't like about the things I like. Well, I'm going to list of my top five Loves and Hates for Drupal 4.7.

Hates

So that I may end on a postive note, I'm going to cover the hates first.

1. The documentation is out of date. This can be said for a large number of development projects, particularly those that are run by a community without the influence of much corporate backing. Drupal isn't used widely by corporations as much as it is used widely by non-profits, special interests, and other associations. That's not bad, it just means that there aren't a lot of QA folks demanding proper documentation.

   I do want to make some clarifications:
   

   
   
   • This issue doesn't extend to the API documentation, which is quite possibly the best documented API of any system I've ever used.
   
   
   • This is partially a symptom of the fact that there is so much documentation and that development progresses so quickly. It's easy for a document to become outdated quickly and not enough eyes to go back and fix it quickly
   
   

   Finally, I'd like to suggest that Drupal might be able to help with this problem if they allowed more freedom with editting the documentation content. I believe this is part of the reason the Drupal community would like to see an integrated wiki module. It could certainly help.

2. it is hard to determine what capabilities are available and which modules work well. Without a PhD in Drupalology, it's hard to know what functionality is available through modules. Furthermore, since there's no evaluation system for modules, it's hard to know which implementations work well. People should be able to state whether they like it or they hate it. It isn't easy to know which module is a hack and which module is a mature product. It isn't easy to tell who is contributing to or leading the development of which projects. It's hard to tell how active an individual project is.
   • Each project must have a single sentence description that is shorter than the teaser provided on the main projects pages. This would help folks find them.
   • Project owners should be able to annotate their project with notes showing the development state: pre-alpha, alpha, beta, released, and mature.
   • There should be a list of all active project participants with a clear delineation of who is in charge of a project and a better indication of orphaned projects.
   • There should be a clear show of updates to the project. When was the last commit? When was the last release posted (shown on the project page, but not part of the summary list)? Frequency statistics would help.
3. Drupal suffers from non-developerism. This has gotten better, but one reason why I usually hesitate to use PHP is because the language is so easy to use. Drupal has done a wonderful job of extending PHP into a very easy to use and easy to understand framework.

   This is fine, but making a programming framework easy to understand has a price: those who aren't programmers will start to patch it and work on it. These folks may be ignorant of the full ramifications a particular action might have. This doesn't make them bad people, it just means their expertise lies somewhere else. It's no worse than me relying on a mechanic to fix my truck while attempting to do some simple things myself. I could cost myself a lot of money if I started messing around with the wrong pieces. I probably wouldn't know I was breaking things while I did it.

   In the case of programming, the cost is the potential for code that is difficult to extend, vulnerable to security issues, and inefficient. A non-programmer may not realize that using nested loops might be a worse idea than serial loops in a particular case. A non-programmer may not realize that using a certain function makes it possible for a malicious user to inject code into the system. A non-programmer may never have heard advanced data-structures to help him choose a better solution through a problem.

   In the case of Drupal modules, I've found several look gorgeous inside when you crack them open. Others look like someone implemented the functionality by copying bits and pieces from elsewhere and then just made it work by brute force. Others look utterly horrible. This is the case with any OSS projects, but I've found the range in quality in PHP projects to be a bit wider than say Perl or Java projects. They're simply not as accessible.

   This is why the previous point is important. It should be easy to determine whether members of the community like or dislike something and why they have those opinions.

4. Taxonomy is underused. Taxonomy is, in my opinion, the coolest feature of Drupal. Using taxonomy, you can carefully control the tags associated with any document in the system. You can set up free tagging (a.k.a. folksonomy) where any keyword or collection of keywords can be associated with a document. You can set up specific hierarchies so that a document must be placed into the tree at some point. You can create relationships between different branches of the tree. You can create synonyms. You can mix an match these in just about any way you can think of.

   And yet, taxonomy hasn't been used to facilitate a number of extensions that could have. The best example of extending the taxonomy is the built in forum system. Determining which post a forum topic belongs to is a matter of checking which taxonomy term it belongs to under the forum vocabulary. Forums can be arranged into hierarchies themselves, which are also controlled by the taxonomy. This is very flexible.

   However, many other modules have chosen not to use the taxonomy. The clearest example of this is the Organic Groups module. This, in my mind, is a clear and obvious case for using taxonomy. Each group is a term in a taxonomy vocabulary. If a site wants nested groups, they can do so by making the vocabulary hierarchical. If they want related groups, they add those relationships in the taxonomy. If they want groups that have multiple parents, they can do that. If they want a plan flat system, they can do that. Other examples include the Book module, the Workflow module, the Voting API, and the CCK.

   The taxonomy system is beautiful and it should be used more. I can only think of two reasons why it isn't:

   1. The taxonomy system isn't easy to comprehend and requires some planning to take advantage of properly. Since it isn't easy, that could be an obstacle, though it shouldn't be.
   2. The taxonomy system is pretty weak. As it comes, it doesn't have anything but the ability to create structure and attach documents into that structure. Which leads me to my next and final hate...
5. The taxonomy system is underdeveloped. This system should have been massively enhanced long ago. There is an excellent improvement to the taxonomy system in the Category module. This enhancement replaces the Taxonomy and Book modules with a new system that elevates every vocabulary and every term to first class citizens: nodes. The Category module allows for much greater flexibility and allows you to treat any term as a node, which has the added benefit that it can now be assigned full-text values.

   In addition, I think there should be an enhancement to the Taxonomy (or Category) to allow for values to be associated with a term. This would allow for an even more flexible way of implementing the features of CCK or the Voting API. In fact, with a combo like this, you wouldn't really need much of anything other than Taxonomy, Node, and Forum. This is probably a little radical, but I really like the idea.

Loves

Okay, now that I've exposed why I don't like it, let's consider why I still use it anyway.

1. It's under active development. This is the most important selling point for any piece of software I use. If there hasn't been a change to the code within the last year, I will look for something else. If there hasn't been a change within the last month, I start to worry. Open source projects should continue to update on a very regular basis.

   Drupal is one of those projects that if you forget about it and come back in a year, you have to take some extra time reading about it to catch up. It's not just that those two or three key things you really wanted got put in since you last looked, but that a dozen new features were added and a couple of them are really cool and you hadn't realized how cool until you realize how they work. That's what Drupal has been like for me over the last year.

   I don't necessarily agree with some of the direction they've chosen, but when do I ever agree with all the decisions other developers make? I, of course, know better than they, but I must be patient since not everyone can be as brilliant as me. :P

2. It's got an active community. There are very few posts to the online forums that go unanswered unless the question is completely out in left field or already answered if they'd use the subject line to search instead of creating a new post (of course, I see a lot of those answered with somebody saying "Already answered, look at these links"). Sometimes the answers demonstrate a completely lack of knowledge, but there is an enthusiastic community.

   In the case of Drupal (by my perception, someone else's might be different), there's not a lot of criticism when someone writes a dumb answer to a question or asks a dumb question. In a way this encourages stupid questions and answers, but it also means that the community is willing to help newbs. This is refreshing, since many OSS communities eat their young (another one I participate, in particular, summarily executes the newbies on a regular basis).

3. Easy to understand/extend/install. You can know just enough to purchase a hosting service and then drop the files into place and you're practically running. If you use CivicSpace, the deployment doesn't even require messing around with any SQL scripts, if I recall correctly.

   After installation, turning modules on and off is very easy. Adding new modules might be a bit of a chore, but once you get the files unpacked and FTP'd to the correct place, that's generally pretty easy too. Configuring modules is also very easy once you find where the options are held (harder than modifying the actual configuration).

   To extend Drupal, you just design a new module or tweak an existing one. This is generally very easy as well. This has the caveats I mentioned above as being a little too easy. However, overall, it's really a cinch to do most things.

4. There are lots of documentation and examples. I mentioned that the documentation is a bit out of date in a lot more places than it should. On the other hand, there's just so much documentation. There are examples showing how to do many common tasks. There's documentation covering several theme customizations, starting custom modules, tweaking various things, etc.
5. The system is extremely flexible. Want to do something with Drupal? There's probably a configuration option, a module, or a module setting that does that. If it's something unusual, you can probably arrange it by combining two or three add-on modules in various ways. If it's way out there, you could surely do it with a little bit of custom development because the module system is fantastically easy to put to use.

All in all, I like Drupal. It's not my favorite CMS system ever, but it's my favorite that is under active development. Now, if only it were written in Perl....

Cheers.

I want to create online communities, content management tools, and collaboration sites through the use of Open Source software, particularly using Perl-language tools.

Education (a.k.a. how much homework I did)

M.S. in Computer Science
Kansas State University, December 2004 Manhattan, Kansas

B.S. in Computer Science
Kansas State University, December 2001 Manhattan, Kansas

Attended
Manhattan Christian College, August 1996 to December 1997
Manhattan, Kansas

Publishing (a.k.a. bragging about my ideas)

Andrew Sterling Hanenkamp, "Developing RESTful Web Services in Perl," ONLamp.com, O'Reilly, //www.onlamp.com/pub/a/onlamp/2008/02/19/developing-restful-web-services-in-perl.html , February 19, 2008.

Andrew Sterling Hanenkamp, "Single Sign-on in Jifty using CAS+ (Part 2)," ONLamp.com, O'Reilly, //www.onlamp.com/pub/a/onlamp/2007/06/09/cas-single-sign-on-with-jifty-part-2.html , June 14, 2007.

Andrew Sterling Hanenkamp, "Single Sign-on in Jifty using CAS+ (Part 1)," ONLamp.com, O'Reilly, //www.onlamp.com/pub/a/onlamp/2007/05/31/cas-single-sign-on-with-jifty.html , May 31, 2007.

Andrew Hanenkamp, ¿Using Java Classes in Perl,¿ perl.com, O'Reilly, //www.perl.com/pub/a/2006/12/21/using-java-classes.html , December 21, 2006.

A. Hanenkamp, D. Andresen, ¿Heterogeneous Channel Bonding Revisited,¿ to appear in the Proceedings of the IASTED International Conference on Parallel and Distributed Computing and Systems (PDCS 2003), pp. 387¿392. Nominated for IASTED Best Paper Award in the area of Communication Issues. Marina Del Rey, CA, November 3-5, 2003.

Skills (a.k.a. Buzzword Bingo)

Bluga Webthumbs Object-Oriented Programming Perl Best Practical RT Visual Basic Ajax Amazon SQS CGI Java FastCGI REST SOAP/WSDL Jifty C++ XML-RPC CSS XSP XHTML SQL JSP/Servlets mod_perl JavaScript XML Prototype PHP XSLT Apache Drupal Amazon EC2 jQuery Trac Tomcat Magnolia SQLite Jackrabbit/JCR Oracle Python Microsoft SQL Server MySQL PostgreSQL Amazon S3 Script.aculo.us C

Experience (a.k.a. companies and people that believed in me)

Interaction Developer
Boomer Consulting, Inc., February 2006 to present
Manhattan, Kansas

• Business Development
  • Participate in strategic planning process at semi-annual firm summits
  • Recommend solutions regarding technical aspects of business plans and ways to profit from technology products
  • Provide technology solutions related to associations, classes, conferences, and consulting provided to clients
• Development
  • Use and customize Open Source content management systems like Drupal and Magnolia ECM to build content infrastructure
  • Use Open Source platforms like Jifty to build custom web applications for handling surveys, statistics, and single sign-on
  • Gather requirements, design, implement, test, and document software development projects
• IT Support
  • Manage dedicated hosted servers by installing and upgrading custom applications and other related software
  • Create server installation images and deploy and maintain server instances using Amazon EC2
  • Create documentation and handle support calls for staff and clients

Systems Coordinator
Kansas State University, October 2003 to February 2006
Computing & Information Sciences Manhattan, Kansas

• Development
  • Customize Best Practical RT for specialized issue tracking
  • Develop a custom CMS for the internal knowledge base
  • Extend Microsoft Active Directory for cross-platform accounts
  • Develop an agent-based configuration management system, integrated with RT
• Communication
  • Teach an undergraduate course on Computer Architecture
  • Help faculty, staff, students in a 250+ system network
  • Write policy, administrative, and end-user documentation
  • Manage four student staffers and one full-time staff person

Graduate Research Assistant
Kansas State University, October 2001 to October 2003
Computer & Information Sciences Manhattan, Kansas

• Use J2EE, EJB, and JSP to create a prerequisite checker
• Configure and maintain Oracle 9i RDBMS
• Linux kernel development for channel bonding (trunking) experiments
• Answer students' system and database questions

Part-time Network Consultant
Network Resource Group, Inc., October 1998 to November 2001
Manhattan, Kansas

• Develop web-based project management system using Java/J2EE
• Extend a Perl-based spam and email antivirus filtering solution
• Build PHP-based ASP tools for monitoring client email filtering
• Develop a Windows GUI application for data-entry in C++
• End-user computing, server, and network support for clients

Projects (a.k.a. pro bono work)

New Hope Church web site

• Install/maintain Drupal installation.
• Integrate layout design and improve content design
• Help manage content and policies
• Develop helps and other materials on how to use the site

Jifty

• Minor contributor to the project
• Specifically interested in Jifty::DBI object-relational mapping API, database backed models API
• Worked on class auto-generation
• Built the initial graphing and charting plugin and API

CAS+ Implementation of CAS

• Provides a single sign-on server compatible with Yale's CAS
• Written with Jifty in Perl

Crossite Module for Drupal

• Allows a single Drupal installation to share/not share nodes on multiple sites
• Makes use of taxonomy to make decisions per node

Promotional Code Module for Drupal

• Allows for registration with a promotional code
• Grants special privileges on registration (or on activation)
• Optionally strips privileges, blocks, or deletes an account when the promotion expires

Presence (a.k.a. other places I appear on the Information Superhighway)

• My Facebook Profile
• My LinkedIn Profile
• My Ohloh Profile
• My CPAN Page

Favorite Books (a.k.a. others sometimes have good ideas too)

A list of technical, non-fiction books I am currently reading or recently read and enjoyed.

• The Myths of Innovation by Scott Berkun
• Open Business Models by Henry Chesbrough
• Perl Best Practices by Damian Conway (I do not frequently agree with Conway, but he does bring up ideas that make you think about how you code in Perl.)
• Innovation Happens Elsewhere by Ron Goldman and Richard P. Gabriel

I've participated in a few Open Source software projects. Some were run by individuals, some by communities, and some by corporations. I'm interested just now in discussing the ones run by corporations. Corporate Open Source holds a somewhat suspicious niche in the market. I'd like to consider here my thoughts on how a corporation can Open Source products and cultivate a community to help develop them.

As a whole, the business community has come to see some value in Open Source. This is particularly true in specific markets where a product or collection of products have really proven themselves, Apache is usually the first to come to mind. I think businesses are quite willing to use Open Source where it is proven, willing to consider Open Source where it looks promising, and willing to develop Open Source where there is potential. However, there is always an undercurrent of skepticism associated with Open Source because the monetary model isn't quite as clear cut as the usual "I'll give you this, if you give me $XXX."

On the other side of things, communities spring up around any successful Open Source project. The quality of those communities is determined by the skills of individuals that makeup of those communities, the attitudes held toward the project, and how well the community works to encourage participation by newbies and gurus (and all the betweens) alike.

Corporate Open Source has a unique position because it melds a more traditional buesiness development model (hierarchical, strict requirements, design, and implementation, milestones, etc.) with the more chaotic Open Source model (chaotic, contributions based upon individual interest or need, few/no requirements, vague deadlines, etc.). If the project is successful, the corporation will also be bolting on a community following which will help direct, design, implement, debug, and test the product. I believe this is a Catch 22 situation: the corporation cannot have a truly successful project without developing the community and the community cannot grow without having the corporation properly directing a successful project.

I see the keys to community development as being wrapped around two key ideas: ownership and communication. I'll start by describing communication and finish with ownership.

Communication in an Open Source project has to be open and honest. This openness and honesty must be percieved more than it must be real. I'm not suggesting that it's a good idea to hide things from your community if you can still make the community think you're honest. I'm telling you that you don't have to dump out the contents of your closets for them if they believe that you are being open and honest---i.e., earnestly telling them enough.

Therefore, keep the marketing people out. At least keep the typical marketing techniques for making your product and company look shiny and perfect out. Just as corporations tend to be a bit mistrusting of the Open Source business model, so are the developers mistrusting the company that has given them something for nothing. The corporation must work extra hard to build trust because both sides will start from a point of skepticism and that skepticism will always remain close under the surface.

Which leads me to my next idea: ownership. You can dispell much of the skepticism of the community by giving them ownership. This doesn't have to be literal ownership of the copyright, but give up enough control in the direction of the product that the community has an important say. If you're planning on a new feature A, but the community is clamoring against feature A to get feature B, consider giving them B first (or instead). If the community contributes a feature that your company can't get any direct value out of, but the community at large loves, consider including it in the main distribution and adding support for it anyway. These are ways of giving the community a sense of ownership. It should be noted, however, that if the community finds a feature valuable, you and your paying clients probably will too.

But why all the trouble? Why would I bother to invest in developing a product just to give it away? The value is that if you develop a "fan base" in the community, you can get improvements, ideas, and testing for free. The opportunity exists for a member of your community to suggest an improvement or provide a patch that links your system up with another system that happens to be a match made in heaven, but that your own staff might not have found while doing research in their spare time---the Internet is a pretty big place after all.

In the case of Boomer Consulting, we sell service. Clients pay us to help them figure out what they can improve, connect them with resources they might not find on their own, and to connect them with other companies who've succeeded in the projects they're still looking forward to completing. I've begun making some contributions to some Open Source projects and we very much plan to make the effort required to release those to the public. These components aren't going to be useful on their own (software never is until someone works to install it, configure it, and use it) and they won't include any of the business logic specific to Boomer that makes us valuable to our clients. We'll be giving away infrastructure that will require considerable investment to really use to compete with---not because the tools are incomplete, but because software doesn't magically start working for you just by installing it.

Yet, if we can get others interested in how those tools work, we can get infrastructure improvements for free when someone writes a patch to do X better or Y to link in another piece of software. All in all, I believe Open Source to be a win for where I work. I'm looking foward to what we can produce during the next year or two.

Cheers.