DSLs are nothing new. Just by encapsulating the concepts of a domain in classes (”domain objects”) and by defining the methods on those classes that act on them (”actions” and “messages”), you create a DSL of sorts. You see this a lot in unit testing. As a body of unit tests evolves, common SetUp and TearDown code is often extracted to avoid duplication. As the extracted classes and methods grow and evolve, a DSL emerges. For an excellent example of this, see Clean Code: A Handbook of Agile Software Craftsmanship by Robert Martin.

Groovy accomplishes the creation of DSLs in a number of ways:

1. Optional Syntax: In Groovy, much of the syntax is optional: semicolons at the end of statements when they are the only statement on the line, parenthesis around arguments in a method call, etc. Dropping away such “noise” is one way that the user can then focus on the domain at hand.
2. Missing-Method Methods: In Java, if you call a method that does not exist, you get a compiler error. In Groovy, it waits until runtime to decide if it’s a problem, giving the class with the missing method a chance to make up a method on the spot. This is done with a special “missing-method method.” For example, this is how GORM allows you to make up method calls like findByPersonAndJobTitle(). There’s a missing-method method that knows how to parse out the name, “findByPersonAndJobTitle” see that it’s comprised of pieces it understands (a find action and two field names) and thereby construct the logic to perform that action.
3. Operator overloading: Groovy allows you to define methods that correspond to the various operators, like plus() for the + sign. This means your DSL could allow for expression like “car = chassis + 4 * tires” and “truck = (cab + 6 * tires) + (trailer + 4 * tires) + (trolley + 4 * tires) + (trailer + 4 * tires)”
4. Named Parameters: Grails has the ability to emulate named-parameters (which means that having to know the order in which to pass parameters is no longer a concern). This is done by defining the method to accept a single map as a parameter. Then, in the call, a map is created on the fly with hard-coded keys that look like parameter names, and expressions that are the corresponding arguments. Leave out the optional square brackets that normally enclose a map constant, and wallah! Named parameters:
5. Closures: Groovy’s strong support for Closures means it’s easy to define custom control structures
6. Categories, MetaClasses, and GroovyObject: Without going into details, these are some of the core facilities in Groovy that make it a dynamic “scripting” language and so suitable for expressing DSLs

Builders:
The most common use of DSLs in Grails is the idiomatic “builder.” This is for when there is a distinct hierarchy, or tree-structure to the elements of the domain. There are pre-defined builders for Swing, XML, HTTP, Hibernate Criteria, HTML, ANT, and the Constraints builder in Grails that drives the validation mechanism, just to name a few. It’s easy to create your own builders as well.

With the groovy.xml.MarkupBuilder(), for example, instead of having to write your own XML…

<stocks>
  <stock symbol='JAVA'>
    <quote day='Mon' price="53.125" />
    <quote day='Tue' price="54.5" />
    <quote day='Wed' price="51.75" />
  </stock>
  <stock symbol='MSFT' />
  <stock symbol='IBM' />
</stocks>

You can express the information like this:

builder.stocks {
    stock(symbol: 'JAVA') {
        quote(day: 'Mon', price: 53.125)
        quote(day: 'Tue', price: 54.5)
        quote(day: 'Wed', price: 51.75)
    }
    stock(symbol: 'MSFT')
    stock(symbol: 'IBM' )
}

At first glance, it might seem like you’re just trading one arbitrary syntax for another: parenthesis, braces, colons and commas, rather than angle-brackets, equal signs, and matched-up start and end tags. However, this is merely a dumbed-down example. The benefits of using a builder are enormous:

For one thing, the Builder syntax is universal. Once you know who to write XML markup using Builder syntax, you also know how to write HTML markup, Swing forms, Grails criteria, and all of the other pre-defined builders mentioned above, and more .

Also, since Builder syntax is built on top of Groovy, Groovy code can be intermixed with the DSL code. For example, we can use each-loops to build the XML markup from a “tickerInfo” object:

builder.stocks {
    tickerInfo.each {ticker ->
        stock(symbol: ticker.symbol) {
            ticker.dailyQuotes.each{
                quote(day: it.day, price: it.price)
            }
        }
    }
}

DSLs are a key reason for the success of Grails. Taking advantage of Groovy’s DSL support is a prime example of how much thought has gone into the architecture Grails with it’s prime directive to be as idiomatic-Grails as possible.

We’ve only just scratched the surface of what DSLs are and how they are supported in Groovy. Here is some recommended further reading:

• docs.codehaus.org/display/GROOVY/Writing+Domain-Specific+Languages
• How Builders Work
• Clean Code: A Handbook of Agile Software Craftsmanship by Robert Martin

Related articles:

• A brief review of Clean Code
• Everything’s Groovy
• IDEs for Groovy/Grails

Schwaber’s first book, Agile Software Development with Scrum, is more of a reference book than a how-to. It describes what Scrum is, but not so much the nuances of how to use it. The second book includes a reference section in the back that pretty much recaps the first book. So, start there. If a pass through the reference section makes sense to you, then proceed directly with the second book. Otherwise, pick up the first book as well and use it to get up to speed.

If you are not yet up to speed on XP, the quintessential tome, Extreme Programming Explained: Embrace Change (2nd Edition) by Kent Beck, is still the best starting place. I’d recommend reading that before reading up on Scrum. As you do, just keep in mind that Scrum essentially replaces and expands upon the XP “planning game.” Don’t skip reading about the planning game. Just understand that if you have questions about how the planning game works in the real world, hold those questions until you get to Scrum.

In Clean Code, Martin doesn’t just name the best practices we should all be following. He explains the reasoning behind each one and gives names to the concepts. Just as the idea of software design patterns revolutionized the way we think and talk about software architecture, Martin’s exploration of day-to-day coding habits gives us a smarter way to think and talk about that.

Case in point: Clean Code kicks off with the practice of giving your objects meaningful names. One aspect of this is that good names do not require anyone who might read your code in the future to have to perform any “mental mappings.” Here’s an example of this that I came across just the other day.

var ld = new LoginData();

Look carefully, that’s a lower-case L. At first glance, it looks like a capital I, as in “ID,” as in “identifier.” I don’t know how long I spent confused about how a class constructor could be returning an identifier when reading through this code for first time. Maybe ten or fifteen seconds? Time in which I was completely distracted from understanding the rest of the code, having pushed that problem onto the stack, as it were, in order to first solve this mini-problem.

Even without the I vs. L confusion, using an abbreviated variable name meant that whenever I came across it, I had to keep reminding myself that it stood for an instance of LoginData. That may not seem like a big deal, but we humans only have so much stack space before we run out of short-term memory. When a program is full of one- and two-letter variable names that must be continually translated mentally, it leaves no room for thinking about the actual problem that the program is trying to solve.

All of this could have been avoided if the name was spelled out to begin with, so I changed it (using the automated rename refactoring tool in my IDE):

var loginData = new LoginData();

“One difference between a smart programmer and a professional programmer is that the professional programmer understands that clarity is king. Professionals use their powers for good and write code that others can understand.” ~ Robert (Uncle Bob) Martin

Martin could have just said, “don’t use abbreviations in variables names, because you’ll make the reader translate it in his head every time he reads it,” and that would have been terrific advice alone. But what he really gives you is a deep understanding of the key principles involved, so that you won’t need to memorize a bunch of arbitrary coding standards in order to write cleaner code. Instead, you’ll be armed with a few simple, understandable principles to guide you, with clean code being the natural outcome.

Clean Code is a must-read for everyone in the software profession — from college freshmen on up to CTOs — especially anyone who finds himself out of work in this economy (or worried about the possibility). Learning to write clean code, and thus to write more valuable code, is probably the single most effective thing you can do for yourself. Programmers are expensive resources. The work we produce is costly. It’s up to us to treat it with the respect it deserves, to try and make it as valuable as possible, and to maintain that value as long as possible.

Groovy is most definitely the star of the Grails show. Every decision that went into formulating Grails started with, “What’s the Groovy take on this? How can we best take advantage of the innate ability of Groovy to solve this?” This attitude makes for a framework that’s powerful, yet minimal and consistent.

This attitude also goes beyond the Web framework. Groovy can be used end-to-end from inside the web server pages to the outer build scripts and even your shell scripts. In the Java world, before Grails came along, one would have to work in seven different languages to build a web app. Grails potentially reduces that to three:

appender.'errors.File'="stacktrace.log"

to

appender.'errors.File'="../logs/${appName}-errors.log"

Determining What is Logged
Config.groovy also determines what is logged. The default logging level is error. For some packages, you’ll want to change the logging level to warn or info. Your choices are, in order: FATAL, ERROR, WARN, INFO, DEBUG.

Grails automatically generates a log4j.properties files from what’s in Config.groovy and places it in the /WEB-INF/classes folder. The log4j.properties file can be tweaked in place as you attempt to debug a particular problem, but just remember that it gets reset every time the app is deployed. So, permanent changes need to go in Config.groovy.

I used to be a big fan of Eclipse, but the Groovy/Grails support in Eclipse is sorely lacking.  Three months ago I splurged on a copy of JetBrains IDEA 7.0 (http://www.jetbrains.com/idea/index.html) and haven’t regretted it one bit.  I’m sitting on 7.0 build 7860 (”Selena”).  Since then they’ve released 7.0.4 (build 7941) and a preview of 8.0 milestone 1, but neither seem to have anything I need, so I’m sitting tight.  I am, however, anxiously awaiting 8.0 milestone 2,  which promises more Grails refactorings and support for Grails 1.0.3. (My projects are still on 1.0.2.)

I just saw an announcement that NetBeans 6.5 (free open source, http://www.netbeans.org/community/releases/65/) is in beta test and that it includes Grails/Groovy support.  For anyone doing Grails work who can’t afford IDEA, I suggest you check it out.  If you do, I’d like to hear from you.  Let’s compare notes.

Getting started with Grails requires two efforts: learning how Groovy extends Java, generally, and learning how Grails then takes advantage of Groovy for rapidly developing web pages. For the former, I highly recommend the brand new book, Groovy Recipes by Scott Davis (Pragmatic Bookshelf). Chapter three exactly and concisely explains what an experienced Java programmer needs to know to get started with Groovy.

After that, The Definitive Guide to Grails by Graeme Keith Rocher is probably where you’ll turn. Be aware, though, that a second edition is pending, and you’ll want to update your copy when it comes out, so budget accordingly.

It’s taking best practices to the extreme…

• Long iterations become short iterations with early experience
• Long, irregular meetings become frequent stand up meetings
• Back-end testing becomes unit testing
• Specification artifacts become stories and a full-time customer (proxy)
• Enforced deadlines become developer-derived estimates
• Code reviews become pair programming

The XP Values

• Communication
• Feedback
• Simplicity
• Courage

The Tenets of Extreme Programming & Refactoring

• Establish a guiding vision, or “metaphor,” and then design as you go, with no big, up-front design.
• Do the simplest thing that works, at first, and then refactor as needed.
• No “Spec Gen” (speculative generality). Rely on refactoring to add features only as they actually become required.
• No speculative performance tuning. Rely on refactoring to introduce performance tuning later.
• Code in small iterations and fast release cycles.
• Put unit and functional testing at the core of the project goal posts, not as an optional add-on. In fact, write the tests before you write the code to be tested. You cannot rely on refactoring without a complete baseline of unit tests.
• Do not refactor and add functionality at the same time. Alternate between them.
• Coding standards should have buy-in by all team members and call for the least amount of work possible.
• Work directly with an on-site customer and/or user and make them a part of the programming team.
• The customer determines the priorities, not the developers.
• Developers provide the estimates, not the customer.
• “Spikes” are written when preliminary research is required to provide a confident estimate.
• The entire development team has collective ownership of the entire project. It is not portioned-out code to individual experts.
• Program in pairs to assure quality and stay on track.
• Use stand-up meetings to stay on track.
• Stick to a 40-hour workweek, sleep well and lead a balanced, healthy lifestyle.

Getting Started

1. Kent Beck’s “Extreme Programming Explained” (second edition) is the seminal work in the field. It’s an easy read and it paints a good overall picture.
2. Martin Fowler’s “Refactoring: Improving the Design of Existing Code” is a must-read. The first 100 pages describes refactoring (and a bit about test-driven development). The remaining pages are a catalog of refactorings — essentially a cookbook of step-by-step recipes.
3. Robert Martin’s “Agile Software Development” has also been recommended as a good starting point, though it’s not primarily about XP.

1. Maintain a written glossary of domain terminology. It’s amazing how often the developers and the customers think they’re talking the same language, but they’re not. Misunderstandings like this are a common source of “assumption errors,” which are a leading cause of wasted effort.

2. Maintain a written “Scope In/Out” sheet. Be explicit about what’s in scope for this project (for this phase of the project) and what’s out of scope.  Jealously guard the scope.  Do it in writing.

3. Make sure everybody who generates and consumes time estimate numbers looks at them the same way. Don’t let an optimistic estimate be taken for realistic or pessimistic one. Beware of using +/- ranges. They are meaningless when talking about software development. Much better is to use degrees of resolution (No confidence/WAG -> Low confidence -> Med confidence -> High confidence). Do a work-breakdown analysis on any large task, recursively, until it is the sum of 2-day or smaller chunks. Any estimate that’s longer than 2-days without a breakdown to back it up cannot be trusted.

4. Create written Use-Cases/Scenarios to communicate user-interaction requirements — especially when it comes to describing exceptions to the rule (things like handling the sale of alcohol or tobacco, processing returned merchandise that’s saleable vs. damaged, the difference between 50% off vs. buy-one-get-one-free, etc.). The Use Case is an amazingly powerful tool for eliciting requirements from a non-technical customer. To start, you have to write the scenarios yourself and get the customers to sign-off, but once they see where you’re going with them, they’ll start writing their own in the first place.

5. Hold daily scrums (stand-up meetings). Each team member should cover three things: (1) What they did since the previous stand-up meeting, (2) What they plan to do before the next stand-up meeting, (3) Any perceived impediments to getting it done.