Being an Agile Architect

I told Steve today that I had been really agonizing over this post and had started it many times but wasn't sure I could word it in ways that I could express what I would like people to understand and take from this post. So here goes and maybe it will make some sense-).

I asked a rhetorical question back in a previous post where I said, "Also, I ponder whether Agile or Extreme Programing development and "Continuous Architecture" and relentless continuous delivery of business value functionality every single week ever really allows time to build adequate framework level stuff like this (logging, etc)."

It was a rhetorical question because I already know the answer having been there when we came up Extreme Programming in the 90's. As someone pointed out in my comments, that dogmatic answer is the canonical "Infrastructure Phase" at the "beginning" of the XP project and to a lesser extent, the mystical "System Metaphor" that we deempasize these days. My experience in the 90's with the XP community was that they were not the world's biggest fans of Architecture and most notably Software Architects. These topics realize perhaps a bias especially with topics like ArchitectsDontCode. At first glance, we got that right. You have to remember that a lot of us were reacting strongly against debacles like CMM and ISO9000 that threatened to take down further software projects in a sea of bindered documentation and hierarchical organizations with the uber ChiefArchitect. For the most part, we were all right about this as, after all, The Source Code Is the Design, not those huge UML Rational Rose diagrams that we churning out that produced zero value but were check-off items in some big process list. Models have no use unless they are essentially are Code Is Model. I learned, kicking and screaming at first, to let go of all those useless things I could not explain why I was doing and focusing on delivering business value not process. Architects were just another big process bump in the road in a lot of places.

Then of course in the Microsoft community, many were just slinging VB1-6 code at the walls with wizards and calling "Architecture" something that either Microsoft told them it was or just what stayed up for a while before crashing. In this scenario, someone just become an "Architect" when they had survived being a developer for a year or 3. In many cases, however, the same kind of biases developed towards architecture leading to lines like Don's immortal definition, "Over 40 and Over confident."

In both cases, biases were developed because of bad experiences people had because of process rather than worth skilled and trained Architects brought to the table. I want to take on my Agile community though because you get this misguided "infrastructure phase" nonsense whenever you raise the issues. Since we don't have very much knowledge at the beginning of a project, Agile, rightfully says, we don't do Big Design Up Front. And we do a great job of that in code. But then we're saying, wait, we are omniscient up front in that architecture stuff and we can just make a platform and then go. Don't get me wrong, we preach the values of Refactoring, Continuous Design and by extension Continuous Architecture and JIT Architecture. I agree with my good friend James Shore with his thoughts on Continuous Design and there is a world of things we can do to keep a design simple, refactored and adaptable to change. That's what we do on our team to a very good level of success.

But it's not enough. We found that a year down the road that there was still a lot missing. Too many "infrastructure" or "framework" things were blowing by in this delirious rush to crunch out 100% business functionality every week no matter what. Someone with Architecture experience needs to perform explicitly the role of Agile Architect and restore these capabilities to a team. It's kind of foolish to say that it was supposed to have been done in the "Infrastructure" phase. I wasn't even at the company then and the infrastructure laid was, well, far from complete to base a real world-class scalable product architecture on. What had we as a team missed in the last 39 week rush? Oh just tiny things like Exceptions, Logging, Security, Service-Oriented Design, Workflow, Scaling to server farms, fail-over, etc. You get the idea. A lot of "cross-cutting" concerns that are like totally essential. The answer is NOT a BDUF Architect coming down from the mount with the stone tablets!! The answer is an Agile Architect. "What's that Jethro? Is that like that X-M-L thang I hear about and all?" Well, Scott Ambler, as well as people like myself, have been trying to educate people on the very real differences with being an Agile Architect. These are our Principles. Look at number one; "Deliver Working Solutions." And if most of that is code, well that means that us Agile Architects CODE ALL THE TIME and SIT WITH THE WHOLE TEAM and not on the mount. We use Agile Modeling. We use Simple Models with Sketches, Whiteboards, Pair Programming rather than Rational Rose UML models because the value-add is the business value we generate in code (Software Is Your Primary Goal) not some UML diagram. I am an Agile Architect.

So Agile teams don't be afraid of us. After all we're guides on the same journey.

Hey, I finally made the Carnival of Agilists!!

My good pal and Agile stir up the masses man, Scott Bellware as usual expresses brilliantly a lot of what I meant better than I could and clarifies a lot of things. I agree.

Comments

# http:// said on 07 September, 2006 12:04 AM

Bravo.  Simply Bravo.

# TrackBack said on 07 September, 2006 01:34 AM

Interesting Finds: September 6, 2006

# http:// said on 07 September, 2006 04:42 AM

Hey Sam,

I dunno if this is really just an architecture issue, or a process control issue.  An architect typically isn't engaged in setting iteration lengths, unless the guy wearing the architecture hat is also the ScrumMaster (or Coach), and who is working with the team to figure out the most natural iteration length.

I remember back when you switched to 1-week iterations.  I thought about something that I had heard Mary Poppendieck say about feedback and control systems, and I remember thinking that 1-week iterations might be too small - too small for me, anyway.

The notion in regards to control systems is that there's a point where the sampling rate for feedback happens too fast, and introduces adverse oscillations in the whole system.

If ends of iterations are the points at which our control systems sample feedback from our development efforts, then introducing them on too high of a frequency might cause them to oscillate adversely, and efforts that we’d spend on other areas – perhaps architectural efforts – are spent instead on providing efforts supporting the frequency of the feedback or in dealing with the oscillations.

I dig what you said way back when about trying to drive a value system into your team where providing business value is the top priority, and I remember thinking that the method that you guys chose was pretty gutsy.  Maybe it might have been equally effective without any process-oriented detriments by introducing that shock for only a temporary period – more as a therapeutic effort than a permanent policy.

We had some issues similar to those that you mentioned in regards to system aspects like logging, transactions, concurrency, scalability, etc.  I don't personally know that these things need to be resolved through /an act of architecture/ or if they are just aspects that developers are aware of and have some knowledge or experience to implement them.

When we leave these kinds of things on the story board for too long, they typically end up taking longer than necessary to retro-fit into the system versus just having taken care of them incrementally.  We now tend to tackle system aspects early in the project – at least those tasks like choosing the approaches and putting some framework classes in place.

If this were to take more than a week, and if we were focused on delivering business value at the end of a 1-week iteration, I don’t know how we’d ever justify taking on the implementation of the aspects.   I think this one way in which an adverse oscillation would appear in a software development process that is predicated on weekly value delivery and feedback.

In my shop, we’re on 3-week iterations.  However, our stakeholder can get a build at any time by clicking a button in CC Tray.  I’ve been wondering lately if iteration cycles and feedback aren’t supposed to be specifically tightly coupled, or if they just appear that way coincidentally in most shops.  Depending on the availability of our stakeholder/customer, we can get feedback every day, and sometimes we might not get feedback for a week.  Somewhere along the lines, our iteration cycles and our feedback cycles have de-coupled (probably due to the facility of generating a release built).

Iterations are still guiding us thematically for three-week sprints, and they’re still affecting the inhalation/exhalation rhythm on our projects, but they aren’t necessarily in master control of the release and feedback cycle.  They do sync up at critical milestones, but they also tend to respond to the stress on the project body at any given point in the race.

# SamGentile said on 07 September, 2006 12:05 PM

Thanks my friend for your always insightful and knowledable guidance. On our situation, I think you are right in the Iteration length of one week drives oscillations because the time is too short and and the feedback points have caused wilder oscillations such that my shock was neccessary. I've actually thought that for 2 months now and have been unsuccessful in getting the Iteration length changed because I feel that Business and our external customer are wedded to it pscychologically and it would cause some degree of anxiety. So, what you said makes total sense and I think its the heart of the issue. So I have talked to Steve Eichert and also to Richard, the PM stakeholder about your ideas and we are talking about them.

BTW, I am somewhat responsible for the Iteration lengths and such because in addition to Architect, I am the XP Coach and the Dev Lead BTW-)

But my other points about Architecture still stand with the Principles of Agile Architecture/Modeling especially in the light of an in-experienced team and needing to provide a lot of design leadership. This was something Jim observed while he was here. I don't think the "cross-cutting" concerns are really dealt with XP IMHO having been there since the begining. It depends on a lot of things; the scale of the system you are trying to build, the experience level of the designers and many other things.

In general, I believe that the principles of Agile Architecture and Modeling have a lot of value add onto the process.

# TrackBack said on 07 September, 2006 03:03 PM

Agile Architect

# http:// said on 07 September, 2006 04:02 PM

> Agile Architecture and Modeling have a lot of value add onto the process

Totally agreed.

# TrackBack said on 07 September, 2006 04:09 PM

Sam Gentile on The Agile Architect

# Dean Michael Berris said on 07 September, 2006 08:03 PM

I am a Software Architect (would like to be called an Agile Architect too) and have been leading a team of developers for quite some time already. I see the insights in your post, and would like to say that I agree with a lot of the things already mentioned. If there's something that a team developing software needs, is someone who might code a little when needed, make sure that the product is being built according to specifications, and making the technical decisions on the architecture and the higher level aspects of the software. Of course, it takes experience to lead a team, and it also takes experience to be part of a team -- and being Architect just because you were elected architect or chosen as Architect I believe doesn't cut it. I would also like to add, that given the Agile way of doing things, looking at the big picture should be someone's job -- and that I believe should be the Architect's.

# TrackBack said on 08 September, 2006 12:08 PM

I am an Agile Architect

# TrackBack said on 08 September, 2006 01:06 PM

Some TDD Links

# http:// said on 08 September, 2006 02:08 PM

<p>If you had one or more stories asking for business value that could only be provided by Exceptions, then wouldn&#39;t the team implement Exceptions using continuous design and implementation? &nbsp;Why would you need some super-developer to do this? &nbsp;Wouldn&#39;t a well-factored code base make up-front anticipation of this need unnecessary? Why would you do this if the customer does not ask for the business value that Exceptions provide? </p><p>The same should hold for Logging, Security, Service-Oriented Design, Workflow, Scaling to server farms, fail-over, etc.</p><p>I certainly believe that regular team white-board modelling sessions are an integral part of continuous design in agile software development. &nbsp;I just do not see why we would need some special role like agile architect to do it.</p>

# SamGentile said on 08 September, 2006 04:07 PM

<p>It&#39;s good to hear from you Steve again and thanks for your perceptive comments. I <strong>understand and agree</strong> with much of what you said. <em>I do think much of our situation is what Scott talked about and schedule pressure of the one-week Iterations</em>. So the fact of the matter is that <strong>business did ask</strong> for Exceptions in more than one business value story and the team repeatedly failed to deliver it because they didn&#39;t have the experience, they didn&#39;t have the framework of the Exception Management Block, etc (and yes Business asked me/us to use the Exception Management Block because of it&#39;s integration with other technologies we use).&nbsp;There needed to be someone with the big picture of how a COTS package like the block architecturally was a fit with what we were doing and laying out a path. However, I<strong> do believe much of our problem</strong> was schedule pressure and what Scott provided and we do keep a well-factored code base and do employ Continous Design. We refactor in every story we do and Jim Shore did spend months here with the team teaching Continous Design. In this case, the business did ask for this as they have asked for Security, Scaling to Server Farms (our banks are already doing this and failing with our previous platform)&nbsp;and others. We have a large enterprise platform that needs to be installed in world-wide banks and there are certain &quot;cross-cutting&quot; things that are required and business has laid out. </p><p>Also, as I said in my post, I still believe in what I said as 1) things were not getting done and 2) I still believe that many Agile people don&#39;t believe in the value that Architects provide. The fact that it resonated with many people all over the Internet that are doing Agile, I think supports the belief that people may think this aspect is missing from XP/Agile and someone needs to keep track of the &quot;big picture.&quot; Personally, I am on a crusade to change my profession and bring Agile Modeling and Principles into the Architecture profession. I think this is a worthwhile goal.</p><p>I could be totally wrong but thank you for adding to the discussion. I would like to continue to iterate this dicsussion [:)]</p>

# Dave Rooney said on 08 September, 2006 05:36 PM

Sam, this is a great "stake in the ground" post!  I have talked for years about how the Agile thought leaders are able to already think intuitively of a system's architecture, but we mere mortals have to really work at it.  We need Agile Architects!

Having an Architect doesn't mean BDUF, but rather it means that there is someone with an overall vision of what a system should be.  I've been fortunate enough to work with someone like that, but even then that person needed "details people" to ensure that the vision could be mapped to reality.

# SamGentile said on 09 September, 2006 12:32 AM

Thanks Dave, that's exactly my point

# TrackBack said on 10 September, 2006 09:09 PM

&amp;quot;Pathfinder&amp;quot;, Rather than &amp;quot;Architect&amp;quot;

# http:// said on 14 September, 2006 04:37 AM

I lead a small develoment team that follows an ad-hoc mix of agile methods.  For example, we do pair-thinking but not pair-coding.  We do detailed design, but only use UML-diagrams or other formalisms when they are actually the best way to describe a concept.

We have stabilised on a scheduling process that has been working quite well for our clients, our proj-management and for development:

3-4 month product milestones. Eg alpha, beta, rc, release, as required.

1 month working milestones

Month 1: requirements, feature planning, and the "gestation".

Months 2,3: feature implemented in 1-5 days, or broken into sub-features.

Month 4: increasing % of stabilisation and bug-fix vs feature work.

The critical period for the architect(s) on the team is the "gestation" (we don't actually use that term formally, but it conveys the idea).  During this phase the various features are designed in enough detail to ensure "Its Going To Work".  During this phase we identify infrastructure requirements, conflicting featues, cross-cutting issues, how to divide the work into meaniful projects to allow developer specialisation etc.  We are nearing the end of one of these phases and I cannot overstate its importance.  We have massaged disparate features into a coherent approach, and found correspondences that will unify our implementations.  Also, during this phase, we convert infrastructure requirements into fully-fledged features and represent them as such to the client.  Without this phase I could not in honesty tell the client we know what to do, how to do it, and how long it will take.  But due to the level of detail involved, it is probably impractical to attempt it for cycles longer than a few months.

During the gestation each developer essentially acts as an architect, depending on skill-set.  After the gestation, we essentially revert to being agile developers and attack each feature in an agile way, knowing that the big picture is under control.  The big picture is reviewed as required in what you might consider an agile-architecture way.  But a large chunk of old-school design and planning is critical to the process.

So overall, we live a combination of the various approached described, and so far its working a treat.

# TrackBack said on 19 September, 2006 04:22 PM

Recognizing Software Design Process on an Agile Development Team

# TrackBack said on 20 September, 2006 12:00 PM

Architetti agili in Team agili

# http:// said on 20 September, 2006 06:02 PM

Very nice commentary by everyone.

All I can say is "Normal Distribution".

Some are of the opinion that most or all developers are extremely skilled (is this also assumed in the "Extreme" part of XP?), and this is simply not the case, even for those with long tenure (I've got some in the next office).  In fact, our friend Normal Distribution says that the majority will be only average (or lower).

I think this is where the "architect" has an opportunity to ensure a minimum level of skill, by laying out what the extremely skilled find patently obvious.  People won't know to use logging, etc. if they don't know how to use it in the first place, and frankly, some don't bother evolving their skills, yet still maintain their jobs, and eventually end up on someone's (your) team.

Architecture is important, yet not everyone is skilled enough to come up with it on their own.  It's not a flaw in the system; it's just "Normal".

# TrackBack said on 21 September, 2006 02:52 PM

The Will to be Good

# http:// said on 01 November, 2006 09:14 AM

Although in my extended experience as an architect in large, complex multidisciplinary projects I have never joined an Agile team, I'm very interested in the issue of 'Agile Architecting'. This discussion is provocative and has certainly some interesting points, but I'm left with a pressing question. In my opinion, architecture is driven primarily by product quality requirements, not by functionality. Consquently, the architect must elucidate the quality expectations of the stakeholders upfront, and make architectural trade-off decisions in order to support these requirements. This is a sharp contrast with the Agilist view of an emerging architecture, and I tend to believe that the word 'architecture' should be replaced here by 'infrastructure'. Although I think that the phrase 'emerging architecture' is a contradiction, I'm quite interesting in the idea of 'Agile Architecting', especially in the context of large, complex, software-intensive systems.

Erik Philippus

Erik.Philippus@improvement-services.nl

# TrackBack said on 02 December, 2006 03:42 PM

Our Agile Project Goes into Ship/Performance Mode

# TrackBack said on 25 January, 2007 03:49 PM

Our .NET 3.0 Enterprise Application and Architecture Shipped

# TrackBack said on 23 March, 2007 11:57 AM

New and Notable 151

# TrackBack said on 29 June, 2007 05:23 PM

New and Notable 176

# TrackBack said on 09 July, 2007 01:57 PM

Arcast - The Agile Architect podcast now online

Leave a Comment

(required) 
(required) 
(optional)
(required) 

Search

Go

This Blog

News

    The content of this site are my own personal opinions and do not represent my employer's view in anyway.

    Profile for SamGentile

MVP

Blog Information Profile for SamGentile

Tags

Archives

Syndication