Hi Stephen,

no real argument with anything you've said.  The term architect is intentionally co-opted and people in IT with that title usually are performing a different role than a Business Analyst. In my experience the Business Analyst finds and defines the use cases and the architect then tries to abstract the patterns out of those use cases and figure out the conceptual and logical designs that will meet them.  As for UML, an application architect might be using UML but I find it pretty painful; it seems to me to be more of a detail design tool, and then useful only if you have to convey the detail design of a very large system to many people.  Personally, I'd rather find the natural component divisions, divide those among the teams and let the teams work out the interfaces and not attempt to pre-ordain them completely.

As to  your question "Is software as a written language that has to communicate to machines and humans different?" I'd also say yes.  Not just from DDD, the pure diversity in programming languages supports that and the drive for languages such as Scala to support Domain Specific Language implementations gives some hints on how that can be generalized.  However, DDD isn't going to eliminate any need for architecture.  Identifying the proper domains is still a hard job and, in particular, good abstractions and generalizations are hard to get right.  If that's left to "iterative refinement" then massive re-factoring becomes the rule and not at the level of continuous re-factoring as a good development process but more at the level of throw everything out and start over if you are not careful. Even worse, in many cases the the re-factoring is too complex for the teams to see the patterns across the individual components already built and the iterative design just doesn't happen as a result.  I'm sure you are aware of the studies that show that errors caught in the design phase are way less expensive to fix than those caught later on (least of all after your healthcare.gove site is unleashed on the public)?  Good architecture practices try to push that analysis as early in the process as possible.  People focus on software development when they talk about Agile but that's usually Scrum or Kanban and that's only a small part of the Agile story.  I've heard people suggest that it's as little as 1/12th of the process because of the way XP divides things up.  I'm not sure I'd go that far, but I do think the "Iterative planning" portion of the process doesn't get the focus at as wide of scale as is needed for complex projects.

Peter Hunsberger

On Wed, Dec 4, 2013 at 7:44 PM, Stephen Cameron <steve.cameron.62@g...> wrote:

Hi Peter,

Thanks for this detailed response. Just a few thoughts in reply.

I don't claim any great expertise, this is just something that has interested me. I liked the idea of UML as a design/architecture tool, but once I started to investigate it I found a completely contrasting point of view about its usefulness for such, and one that I have moved across to, specifically that 'the code is the design'.

I have no argument on the need for planning to coordinate the activities of many people with different expertise in large projects. I do question the use of the terms 'Enterprise Architecture'  and 'Software Architect' to describe it. I am much more happy with the concepts of a 'Business-Analyst' as someone concerned with people and process (workflow etc) and 'Software (Systems) Engineer' or 'Information Technologist' as someone concerned with managing information and how to build IT systems to do it.

To me this is naming of roles is akin to the 'architect' and 'engineer' in building construction, as those responsible for vision and implemenation, but more appropriate to IT. They are similar partners (basically what and how), most innovative buildings are a combined effort of architect and engineer, such as FLW's Falling Water with its cantilevered decks. Building Architects often do take a management role in implementation true, but on larger projects the detailed project management is done by engineers, with architects just having oversight. This can lead to problems, there is a classic case of this with the Sydney Opera House, where the architect Jorn Utzon (http://en.wikipedia.org/wiki/Jorn_Utzon) left Australia in a huff before the building was completed, never to return. The building was completed with some compromise of its architectural vision by equally talented engineers under another Dane, engineer Ove Arup (http://en.wikipedia.org/wiki/Ove_Arup).

But my key point is this: that the engineering part, finding a good solution to a complex problem, is itself a creative endeavour, one that might require considerable experimentation with alternative approaches (or refinement of) to find a workable or optimal solution. I think this is not recognised in the general population and, that as a result, unrealistic expectations lead to project failures.

The kind of input from builders that you describe is essentially the 'pattern language' of Alexander, that by experience builders  gain knowledge of what works well for specific scenarios.

My question is: Is software as a written language that has to communicate to machines and humans different? The Domain Driven Design (DDD) school is saying yes, that because written language is so important, iterative refinement is the best way it can be done, like a dialogue between book writer and editor, the film analogy a spoken word and visual dialogue between actors and director and in DDD between software designer/engineer and domain expert mediated via working prototypes. This is Agile basically, but for me more insightful about why and hence more scalable.

In DDD I suspect that role of plans or architecture documents is lower, and where necessary is probably best provided by being generated from the code as an evolving model (and the 'point of truth').

I think I am on bit of a soap-box here. These discussions do go off on tangents easily, but interesting things fall out. Regarding "coordinating up and down those teams and layers of architecture can still be an agile process.": I need to look more closely now at Zachman and Scaled Agile. Also, I don't see the XML messages as being the issue, yes they where slow to arrive, more that what was in them was not reliable, or open to interpretation, so people resorted to phoning the applicants to validate the XML information content, blaming XML would be "shooting the messenger".

Looking wikipedia I find that: Etymologically, architect derives from the Latin architectus, which derives from the Greek arkhitekton (arkhi-, chief + tekton, builder), i.e., chief builder. No help for me there!

Regards
Steve

On Thu, Dec 5, 2013 at 9:49 AM, Peter Hunsberger <peter.hunsberger@g...> wrote:

Stephen,

I think you missed both my points almost completely...  The first is that a good contractor is not constrained by the architect,  the team I worked with that summer was far more experienced at actually implementing the process of building houses than almost any architect.  In fact, many of the architects that worked with them on a regular basis would leave many of the details you touch on unspecified simply because they knew what was uncovered on the job site was better handled by that team.  We actually built a show case house for an architect (his primary residence) where we had to talk the architect into changing several of the "features" as we went simply because they would have resulted in things like windows that were highly prone to breakage, leaky roofs and unsafe balconies. I had the pleasure of touring that house when it went for sale many (10 or more, hard to recall) years after it was built and it was in great shape. If Frank Lloyd-Wright had used (and listened to) such a construction team his houses would be a joy to live in. The analogy holds with software development teams; architecture informs the teams, good teams know when the architecture should be modified.  In application architecture we talk about logical and physical architecture as separate things because of this; the logical design might be implemented with several different physical systems all with varying degrees of success.  Depending on the constraints in the logical architecture you may have constraints on a physical database model or not; depends on the goals that architecture had to meet. If LEAD Gold is the priority you're probably building a very different house than Falling Rock.

The second issue is how to scale this.  The contractor I worked with could not have built a skyscraper and the shared on site experience and knowledge could not exist across the size of team needed. That's when the extra layers of architecture start to apply.  You need the people who do community planning and know exactly where the sewer line has to go.  You need the model builders who model the site in advance for drainage and wind and sun.  You need the city planners who make sure the skyscraper isn't going to screw up traffic flows for the next decade.  Same thing applies to large scale IT.  If you don't have those layers of plans in place you should not be surprised that the 10 independent teams built something that does not exchange XML properly between the individual components.  

But back to the point that drew me into this thread; coordinating up and down those teams and layers of architecture can still be an agile process.  That's why I point at frameworks such as Zachman and Scaled Agile and not at methodologies such as TOGAF.  The former don't specify how development is cycled, that's part of the architectural decision process.  The latter are pretty much restricted to waterfall by their own internal structure.  Large organizations have to use architecture as a way to ensure good communications and to ensure that the right problems are addressed.  You need architectures to ensure the right people are hired for the right jobs and that the right teams are put in the right place at the right time.  The trim carpentry team from my summer job couldn't build the sky scraper but they sure could do a bang up job on finishing out the wood work in the executive suites and board room and they would have been more than willing to engage the lead designers in weekly or even daily dialogues about what was going to work best in each room as they encountered the plumbing, air vents and support columns that were in the wrong spot...

Peter Hunsberger

On Tue, Dec 3, 2013 at 7:35 PM, Stephen Cameron <steve.cameron.62@g...> wrote:

Hi Peter, you are lucky 100th posting on this thread.

However what you say is ringing alarm bells for me.

To me using the word 'architecture' is misleading, Frank Lloyd-Wright is one architect that famously never consulted with his clients, he just gave them a design and asked for payment, as a result many of his houses are actually bad to live-in, though visually charming (to me).

In my view, the only 'architecture' in software is the structure that you build into your software, either in terms of your classes using object-oriented programming, the modules in procedural, or the entities of your data-model.

Your building reorientation analogy misses the main issue, namely was the house, once built, fit for purpose? Did the change in orientation destroy its carefully calculated heat-balance, such that it dropped out of rating 5? Could people find the front door? Was the sewage system always getting blocked as there was not enough fall on the pipe from house to street? Yes, a show-stopper issue was solved, but what were the long-term consequences?

Aren't enterprise architects actually business analysts doing systems-analysis? In the context of an 'Enterprise' they can design processes that aid in maximising profits or other measureable goals. But this to me is a separate specialisation, actually one where a different use of the word 'model' applies than to that in software, its something that is used to test the *likely* impact of changing something in the real system, kind of like economic modelling and forecasting.

My thinking is that the process of creating software that extends the capabilities of humans and organisations must be primarily concerned with the link between humans and computing devices, such as: is what people think is happening actually happening,  is the information obtained in a human-device iteraction reliable, as a new user how do i learn about software efficiently (hopefully by interation with the system itself rather than having to read manuals) etc. I believe that the means to achieve this does involve iteration and refinement with user involvement, and lacking this is a major cause of problems, thinking that a optimal solution can be 'procured' (or imposed). Users 'cope' with this approach nothing more, unless what is procured is so defective that the whole thing comes crashing down (ala ACA).

My fear is that the business analysts want to be software designers (forget the word programmer) as well.

Steve

I don't think we are disagreeing greatly, I am asking for a change of perspective.

On Tue, Dec 3, 2013 at 2:11 PM, Peter Hunsberger <peter.hunsberger@g...> wrote:

The architecture and building construction analogy is useful at an IT architecture level.  At an implementation level it is may be more questionable. However, consider an house architect and foreman working hand in hand; the construction crews are scheduled iteratively in a sense if you consider  framing, wiring, plumbing, dry-walling,  finish carpentry, tiling, painting, etc. all as different development requirements that build on prior results. In the end successful IT projects still need good architecture whether they work with Agile or waterfall implementation methodologies.  The alternative is that requirements are gathered without guidance, from whomever the development team wants to talk to (maybe multiple times on each dev cycle) and the result may be a directionless mashup of 100 or 10,000 conflicting requirements with no real end point ever recognized, least of all met.

When introducing architecture to development teams who fear they may be overly constrained I sometimes relate a story from a summer job in construction in high school.  I was working on high end house construction with a local crew and we arrived at a new site ready to begin.  The very experienced foreman surveyed the site and examined the plans from the architect and made a call to the owner.  He explained that if we built the house as designed any significant rainfall would flow into the garage and flood the basement.  We ended up building the house flipped on the site, holding the plans upside down to figure out what went where.  Some mistakes were made as a result, but waiting on a new architecture iteration would have been more costly than fixing those mistakes as we went...  We couldn't have built the house without the plans, but practical implementation was not pre-ordained by those plans.

A lot of what is being discussed in this thread, is in my experience, preventable with good IT architecture.  The analogy to construction holds at three levels: city planning, neighborhood plans, and individual buildings. The business equivalent being enterprise architecture, systems architecture and application architecture. Real enterprise architecture is rare in my experience but the best way to tackle disparate organizations with conflicting goals. At the Enterprise level I prefer the Zachman Framework: fill out who, what, when, where, why and how at the business goals, conceptual and logical design levels and you know you have plans for all aspects of the all the systems, people, geographic areas, etc. Drill down into physical design and implementation plans as needed. Doing that _is_ costly and time consuming.  Not doing it is what leads to the problems discussed in this thread.  If you know what you are doing you can skip filling in the entire grid, but that should be a decision that has buy-in from all the parties involved.  Note that at this level Enterprise architecture is more than just IT, it looks at what people are involved in all parts of  the business, in what geographic areas, when they need to be involved etc. and lays out the products they should deliver and the rational for doing so.  IT may not even be involved in some cases (but those are not the projects that interest me!)

There are also Agile equivalents for Enterprise architectures. In particular, I like the "Scaled Agile Framework", which uses things like Scrum and Kanban type push / pull methodologies at the architecture level and details how to couple them to the implementation process in a traceable and iterative fashion.  I'm not yet sure it works when you are trying to couple things like Amazon like planning horizons (5 to 10 years) to next weeks dev push, but I am starting to examine how to bridge that gap...

Peter Hunsberger

On Mon, Dec 2, 2013 at 9:52 AM, Michael Sokolov <msokolov@s...> wrote:

On 12/02/2013 10:24 AM, Thomas Passin wrote:

On 12/2/2013 2:09 AM, Stephen Cameron wrote:

I think you are getting to the heart of it, but the analogy I like is
making a movie, the final result is very open-ended and coordinating the
activity of getting a result very complex, also, whether the end result
is a success is very much dependant on intimate knowledge of the humans
who will view (use) the outcome.

I think that a better analogy is a large construction project like an office building or a large bridge.  They often run over budget and schedule, and often develop unforeseen problems.  Some of them are almost exactly like previous ones, and some have many new elements. During construction, unexpected problems arise, and once completed, usage patterns may turn out to be quite different from those anticipated.  Many of the implementation details are routine and implemented by tradesmen who don't know much about the overall architecture.

I've operated with that idea for many years, but I'm coming to believe it's actively destructive because it leads to pure waterfall thinking: design, architect, engineer, build.  Where is there a place for an iterative design, build, test cycle in building construction?

-Mike

_______________________________________________________________________

XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@l....org
subscribe: xml-dev-subscribe@l....org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php