Case in point: On a project where we were upgrading a system from Java 1.4 to Java 5, I happened to be the first developer to check in code with Java 5 type safety notation (which uses angle brackets) in a file that happened to be processed by a Hibernate 2 Xdoclet parser during the build process (if necessary). I didn’t realize that the Xdoclet parser didn’t know how to handle those angle brackets. Furthermore, I didn’t notice the problem because of the way the Ant script was set up, it didn’t manifest until after running a CLEAN task, but I had no reason to run CLEAN (just BUILD). So, instead, an unsuspecting developer on the East Coast was made to suffer, puzzling over the problem until I came online three hours later and realized what had happened.

So, in my experience, code generators seem to be much more successful when used as tools that are invoked on demand by the developers. The key distinction is whether or not the generated code is committed to version control. Looking back over the last seven projects that I’ve been involved with that made heavy use of code generators, three of them did so with post-commit generation. The four projects that used on-demand generation always seems to work out much better. On-demand generation simplifies much:

1. The build process is faster and more direct, with less to go wrong at build time, which is beyond the control of the committing developer. This is especially important when automating the build process to be unattended, with tools like Cruise Control.
2. Newbie developers do not need to be aware of the existence of the code-generation tools at all (until their first assignment that requires one). That’s less to go wrong when setting up a new development environment.
3. Exceptions to the rule can be handled manually. No need to spend hours perfecting rickety RegEx replacement tasks in Ant. (By all means, utilize RegEx replacement tasks when it is cost effective to spend the time writing them, but this way you get to choose when that is, rather than being forced into it. Also, this way it won’t kill you if the transform is not perfect.)
4. There is high visibility of the generated code. What’s checked in to version control is what you get. The exact changes are logged and can be easily differenced/compared/branched/tagged, just like any other code. There’s no question as to what the compiler is “really” going to compile and deploy.
5. By putting the developer in the loop between the code generation and the commit, it gives Eclipse a chance to chime in about errors and warnings.
   Also, if Eclipse can see the generated code, that means that all the power and might of the searching, navigating, and refactoring tools come into play.
6. When the code generators make bad decisions (e.g. when presented with a new kind of input), the results are more easily noticed because the developer facing the problem is the one who just did the on-demand code generation. Chances are, the problem will manifest as a blatant error, warning, or hint, or as unit test failure. Even if not, the committing developer might catch it still if he/she is disciplined in comparing the differences of every single file that he/she is about to check in (a good practice anyway for catching things like typos, unaddressed TODO’s, and extraneous debugging logic).
7. It’s easer for developers to mix and match tools, applying them in any arbitrary order, as opposed to the ones that are programmed in to the build process always having to be last.

I admit there are some downsides:

1. Having to make sure the developers know when and how they are supposed to use the tools, at the risk of them thinking they need to do the work by hand.
2. Remembering to revisit previously generated code whenever a code generator process changes, and either manually affecting the changes, or regenerating that code and merging back in any modifications that were done by hand. (Fortunately tools like WinMerge and Beyond Compare make this easier.)

There are undoubtedly more cons (as well as pros), but I’m convinced that the cons are easily managed and that the “low road” is the easier route.

Jim Fleischmann is a security consultant who audits companies for Sarbanes-Oxley compliance [defined]. He and I recently compared notes. As a software developer who has often worked on role-based security systems, it was wonderful for me to get his perspective on where the rubber meets the road. As you might imagine, his feedback was quite different from the usual feedback I’d get through channels (QA bug reports, customer service support tickets, etc.) The following is my recollection of what we covered. I’ll start with Jim’s observations, and then follow up with my own.

I opened the conversation by lamenting on how difficult it is to find information on best practices for role-based security. Jim agreed that there is a dearth of information for this. He suggested that if anything new does surface, I might find it by searching on the phrase “segregation of duties,” that being the overarching concept for which role-based security is just one manifestation.

I next asked Jim what are the most important aspects of any of role-based security system, to him as an auditor. He named four:

• Reporting Capability — The audit stops dead if Jim cannot be provided with hard-copy reports that list: (a) all of the available permissions and their definitions, (b) all of the defined roles with the permissions assigned to them, and (c) all of the active users and their roles.
• Usage Auditing — Next, Jim looks for anything that can tell him whether or not permissions and roles are actually being used. Jim says one of the most common recommendations that come out of his audits is to clamp down on roles and permissions that are not being used. Say, for example, that a user is assigned to a role that is allowed 20 different permissions, but that particular user only ever takes advantage of three of those permissions. Jim’s recommendation would be to define a second role with just those three permissions (or at least a much smaller subset of the 20), and then reassign the user to that new role. Therefore, anything the software can do from an auditing perspective to support the discovery of this kind of information is a boon.
• 100% Role-Based — It’s a red flag to Jim if the software allows a user to be assigned a permission directly (without having to go through a role). That defeats the whole purpose of having roles, he says, and having the roles be meaningful.
• Segregation of Duties Enforced by Mutually Exclusive Permissions — The whole point of role-based security is to give workers access to only the functionality they need to perform their jobs. So, a main focus of any audit is to look for certain tasks that should always be mutually exclusive (e.g. the person processing an order should never be the same person who handles the cash), and then to see that the roles and permissions, as defined, enforce this. All the better if the software actively supports the concept of mutually exclusive permissions, preventing in the first place any one role from being granted both permissions. Furthermore, if the system allows one user to be assigned multiple roles, then the software should also prevent any role combinations that would grant a single user conflicting permissions.

I also wanted to know if Jim had any thoughts on whether or not users should be allowed to be assigned more than one role. Similarly, did he like the idea of having super-roles that are made up of sub-roles, or if that muddled things. Jim did not have an opinion on either. Whichever of these approaches works best for the application, would be fine by him. He did point out that it should be absolutely clear that assigning a user to multiple roles means they are being granted the union of all those permissions, lest someone in administration think they are only granting that user the intersection of them.

Finally, here are a few of my own observations on the practical implementation of role-based security:

• Us Programmers are Lazy — We tend to write code that is wide-open and worry about clamping down on security last, if at all. So, the more we can do as software architects to make security awareness part of the development process, the better. This could mean utilizing code generation and code templates at development time, adding static or dynamic code validation at build time, or taking advantage of aspect oriented programming such as described by Rick Hightower here: www.thearcmind.com/confluence/display/SpribernateSF/ Using+Spring+AOP+to+add+row+level+security +to+Hibernate+or+any+ORM+really.
• Always Code to Permissions, Never to Roles — Application programmers should only code their security checks against permissions, not directly against roles. Roles should be treated only as a mechanism for bundling permissions. For one thing, it simplifies the application code if security checks are always only against permissions. But more importantly, coding directly against a role in effect creates a phantom permission, and it would be all too easy for that aspect of the role, that phantom permission, to go undocumented.
• Hyper Specific Permissions — New permissions should be created at the drop of a hat. Reusing an existing permission for a second purpose is usually wrong. In any event, it’s far easier to merge two permissions that are later determined to be redundant, than it is to split one that is determined to be overreaching.
• Careful Permission Naming — The naming of permissions is hard, but worth every minute spent agonizing over getting them right. It is especially important to ensure that permission names are unique with all other entities throughout the code base. Since permission names are often cited within quoted strings, in XML files, and in database load scripts, most of these occurrences are therefore out of the reach of any of the “safe” refactoring tools that are programmed into modern IDE’s. So, having absolutely unique permission names makes it easier to rely on an ordinary search-and-replace tool.

For more details, see the press release at codejacked.com/sitenews, or simply visit the site itself to see what it’s all about.

Keep your developers focused on what they do best — directly applying their hard-won knowledge of the problem domain — and allow us to take care of the nagging incidentals. We bring expertise in all of the following areas:

• Build Process Automation
• Deployment Process Automation
• Version Control Integration
• Automated Regression Testing (”Smoke Tests”)
• Project Wikis
• Code Generation & Templating
• Static & Dynamic Source Code Validation

We work well with teams of any size, whether they are collocated or distributed, on-site or remote. We can start with whatever processes you already have in place, or create new processes from the ground up. Every job is fully documented and completely transparent, so your developers will have no trouble adopting and perpetuating the chosen solutions.

Questions? Call anytime to learn more or to schedule an exploratory review.

One trick to filtering out the noise is to define a shell script that uses Rsync to create/update a searchable shadow copy of the working folder, and then to search that copy… In case you’re not familiar with Rsync, it is a tool intended to keep two remote file systems synchronized. Rsync’s main claim to fame is that it’s fast because it only transmits the differences, but Rsync is also quite powerful when it comes to specifying exactly which files and folders are to be synchronized and how. It’s this secondary feature of Rsync that allows us to filter out the noise. There are two parts to this solution: the actual shell script, and a file that lists all of the inclusion and exclusion patterns. (This example uses CygWin, running on a Windows box.)

Here is the (entire) shell script (C:workcmdsearchcopy.sh):

 #!/bin/sh
 pushd /cygdrive/c/work
 mkdir -p /cygdrive/e/work_search
 rsync -vrut --filter='. /cygdrive/c/work/cmd/searchcopy_filelist.txt' alpha bravo charlie /cygdrive/e/work_search
 popd

• /cygdrive/c/work is your working folder (that’s CygWin speak for C:work).
• Alpha, bravo, and charlie are the folder names of the projects that you are interested in.
• /cygdrive/e/work_search is the name of the searchable shadow copy you want to create/update (over on your E: removable USB drive).

Here is (an abbreviated version of) the filter file (C:workcmdsearchcopy_filelist.txt), to give you an idea:

 - .svn/
 - bin/
 - build*/
 - deployment/
 - lib/
 - log/
 - .#*
 - *.[ehjstw]ar
 - *.[Bb][Aa][Kk]
 - *.doc
 - *.[Ee][Xx][Ee]
 - *.gif
 - *.httpunit
 - *.ico
 - *.jasper
 - *.jpg
 - *.library
 - *.log
 - *.[Oo][Ll][Dd]
 - *.pdf
 - *.[Zz][Ii][Pp]

In this case, they are all exclusions (leading minus sign), Thus, everything in the alpha, bravo, and charlie folders will be copied, except files or subfolders matching these patterns.

Tips for using Rsync:

• Don’t waste time with the –include and –exclude switches, they are merely dumbed-down versions of the –filter switch, so just use the –filter switch right off.
• Avoid the –cvs-exclude switch, if you can, and pay close attention to what it ignores if you can’t. For example, it ignores any file or folder named “core”, and it ignores *.script files; both of which burned me when I tried using it on a certtain Tapestry application.
• Most implementations of Rsync are case sensitive, including CygWin’s! So if there is a possibility of filenames that exist with multiple casings, then you either have to repeat the pattern or use the square bracket notation:

   - *.EXE
   - *.Exe
   - *.exe

  or

   - *.[Ee][Xx][Ee]

• Pay close attention to the man pages that describe other aspects of the pattern matching algorithm. For example, leading and trailing slashes each have special significance.

I’m helping out with a study group for “Head First Design Patterns,” which just finished chapter 6. On the whole, it’s a pretty good introduction to software design patterns — way more accessible than the seminal work by the Gang of Four; however, the examples sometimes make my head hurt. I can’t imagine what they’re doing to the heads of the beginners in the group. Coming up with decent examples is the hardest thing to do in expository writing, and I certainly give the authors an E for effort in creativity, but I wish they had been a little less concerned with making their examples “hip” and a little more concerned with making them appropriate.

To wit, the whole pizza store analogy in chapter 4 (to illustrate factory method and abstract factory) is flawed. For one thing, that’s just not the way you’d model a pizza business in any actual software that I can imagine. For another, the differences between a New York pizza factory and a Chicago pizza factory are too subtle/trivial to make for an effective illustration of why you would need to subclass anything (much less use a factory to manage the subclasses). A much better example, as everyone in my group agreed, would have been an application that needs to offer up a consistent set of functionality to users who are accessing it in wildly different ways: one’s in a web browser on a desktop, another is running a cell phone app, another is using a touch-tone phone, and yet another is using a voice-activated headset. All the client code knows is that, for example, it needs to ask a multiple-choice question and obtain the answer. It’s up to an abstract factory to provide the client with a set of classes that can do that, in the context of the selected user-interface, in whatever way is necessary.

To a lesser degree, the Starbucks coffee example at the beginning of the book suffers from the same too-hip-to-be-effective syndrome, although I do think that the remote-control example for the Command pattern in chapter six is dead on.

For any novice who is reading this book without the benefit of a study group, I highly suggest that you find at least one other programmer who is experienced in design patterns to explain why/if/how the examples are lacking.

In the field of cognitive psychology there’s this so-called “Magic Number 7.” Basically, the idea is that humans can only keep 7 disjointed “things”, plus or minus two, in short-term memory at once. To see what I mean, study the following list of words for a minute. Then, turn away and write down as many as you can from memory:

Apricot, ladder, storm, headphones, spark-plug, sneaker, anchor, coin, library, twenty-seven, lyrics, nail, telescope, onion.

How’d you do? I could only recall nine of them (even though I was the one who made up the darn list just now), and in fact, I had to work hard to recall the eighth and ninth. By the way, IQ has nothing to do with this number. Whether you’re closer to Mensa or Densa, your short-term recall factor will still be 7 +/- 2. (See Wikipedia and George Miller’s 1956 paper that first noted the phenomenon is at www.musanim.com/miller1956/.)

As the Wikipedia article points out, Ed Yourdon first commented on how this relates to computer science back in 1979. He described what we would today refer to as the long-method smell (see www.soberit.hut.fi/mmantyla/BadCodeSmellsTaxonomy.htm), citing the magic number 7 as why it’s bad to tax the short-term memory of someone trying to understand a piece of code.

So, how often does your web framework require you to keep track of more than seven things at once? Take, for example, the simple task of creating a data-entry page. How many different files need to be created or modified to accomplish this? One for the HTML template, one for the page logic, multiple files for each model object (interface, impl, and ORM mapping), the DAO (interface and impl), the config file with the URL construction rules, the config file with the user security rules, the config file that lists all of the pages on the site, the interface file with constants for all those page names? — That’s eleven and counting. Sound familiar? This amount of complexity often leads to things falling through the cracks, and is usually reflected in a high number of bug reports.

From what I have seen, Ruby on Rails has made this particular task as simple as it can be, and that accounts for a great deal of RoR’s appeal, I’m sure. Even .Net has this down cold. In the case of RoR, it’s an innate feature of the framework. In the case of .Net, it’s simplified by the tools. I’m still waiting for a Java-based framework/environment that “gets it” in this regard.

The best part is that the talks were not all technical. Much of the conference was business oriented, putting the technical aspects into perspective. For example, Abhijit Gadkari opened up his talk about Software as a Service (SaaS) by pointing to where he believes it to be on the hype cycle (half way between the technology trigger and the peak of inflated expectations) as compared to where four technologies are that SaaS relies on: XML (plateau of productivity), web services (slope of enlightenment), SOA (just past the peak of inflated expectations and dropping), and Workflow (just recently triggered).

Jinesh Varia, the Evangelist from Amazon showed off S3 and EC2, which are the greatest things to happen to remote hosting since fibre optics.

Waleed Abdulla presented his XRules project – a clean, straightforward way to encode business rules for working with XML packets, to validate them, and to perform computations on them. The idea is that the same business rules work for local validation as well as remote. He’s written .Net implementations that run on Windows servers and in IE, but the concepts are universal. His use case is that he’s aggregating information between hundreds of different vendors — regional trucking companies if I recall correctly — each with slightly different criteria for the jobs they’ll accept and how they charge for them. This is something that’s going to come up more and more often as smaller businesses figure out how to join forces (and new startups figure out how to do that joining). I’m impressed with what Waleed has done. The Java community would do well to consider a port of XRules.