Agile Programming for Macro Validation

The standard method of macro programming and validation does not work. It is based on the Waterfall method (did you know there is a method that it was based off of??), and is really outdated. The Waterfall method is great for projects with junior level programmers who operate in a field where the requirements are generally fixed. Does this sound like our world? I didn't think so.

The basic idea flows like this:

1. Collect user requirements
2. Translate user requirements into functional requirements
3. Create design documents
4. Create code
5. Test/Validate code
6. #4-#5 repeat as necessary
7. Create User Guide
8. Provide Training
9. Roll out macro
10. Maintain

Anyone working in the pharmaceutical industry as a SAS programmer know that basically you never get to #9. Inevitably, you get stuck because you didn't get all the requirements right, or you got lost in the validation. The first company I worked for, spent 3 years creating a macro that made AE tables. They used the Waterfall method, and never got to #9. Or when they did get to #9 the product did not match the user requirements, but there was no budget to fix it, so in the end they created a product with a lot of documentation that was validated with very low business value.

I propose that we change to Agile Methodology. The basic ideals of Agile programming are:

• Customer satisfaction by rapid, continuous delivery of useful software
• Working software is delivered frequently (weeks rather than months)
• Working software is the principal measure of progress
• Even late changes in requirements are welcomed
• Close, daily cooperation between business people and developers
• Face-to-face conversation is the best form of communication (Co-location)
• Projects are built around motivated individuals, who should be trusted
• Continuous attention to technical excellence and good design
• Simplicity
• Self-organizing teams
• Regular adaptation to changing circumstances

How can we apply this to Macro development?

1. Collect User Requirements
2. Determine the smallest subset of business beneficial requirements
3. Code #2 subset
   
4. Test/Validate
5. Rollout
6. Iterate #1-#5 until all continually changing requirements are met.

So in the development of an AE macro it would be something like this:
Iteration 1:
Requirement: Create counts for SOC/PT table
Code: Very simple, could be done in 1 week
Validation: Also very simple, could be done in 1 week
Business Need Met? Yes

Iteration 2:
Requirement: Allow for an option to specify multiple By groups (Treatment, Relatedness)
Code: Very simple, could be done in 1 week
Validation: Very simple, could be done in 1 week
Business Need Met? Yes

Iteration 3:
Requirement: After using the macro, the user realizes that he needs to be able to make the tables for multiple levels of AE terms (SOC/PT/LLT)
Code: Simple, modification takes 1+ weeks
Validation: Simple
Business Need Met? Yes

Iteration N:
Requirement: New thing a user thought of after having actually used the macro:
Code: Simple to Complex depending on request
Validation: Because it's done iteratively validation is never ONE huge task, it's just many smaller tasks. This also allows the validator to concentrate on creating correct and important test cases.
Business Need Met? Yes, and quickly. Instead of users waiting for months or years for perfect software to come out, they get pieces of useful time saving software frequently.