chris horn's blog

Professional investors understand liquidity. They understand asset backed securities, and they understand the risks when assets subsequently emerge to be poorer quality than they were represented to be. Sub-prime assets can be embarrassingly illiquid and career changing.

In the world of enterprise IT investments, customers have likewise yearned for liquidity. Lock-in to assets available solely from any single vendor implies significant risk to the purchaser. Bad investment decisions into IT assets which subsequently emerge to be poorer quality than they were represented to be, can be embarrassing and career changing – particularly if the assets are illiquid and difficult to replace.

Financial markets have been driven by liquidity. However by contrast, purchasers of IT assets have found it challenging to be able to subsequently replace and substitute alternatives when desirable.

Until now.

Today, I believe that the software and hardware industries are fundamentally changing in favour of liquidity. Software is increasingly componentized. Software has increasingly recognized industry standards which facilitates substitution of alternatives. Open source provides liquidity through lowering the cost of change.

Enterprise software vendors should be trusted partners in providing and maintaining tailored solutions. The ability to successfully integrate a variety of components from a variety of sources to an enterprise level of service is valued. The ability to scalably manage multiple configurations, and evolve them dynamically over time, is valued. Tailorable, personalized solutions for specific customers, partners and staff, but all as part as of the holistic enterprise, are valued. Dynamic systems enable liquidity amongst software assets – no matter from which particular vendors specific assets are obtained.

New, and sustainable, business models are emerging from software vendors who deeply understand technology liquidity.

Single, monolithic, vertically integrated silos of software stacks is thinking from the last century. Integrated stacks are illiquid if any specific components or layers cannot be readily substituted by better alternatives on the market today, or which may emerge tomorrow, from any vendor.

It does not take an oracle to foresee what will happen if BEAS are purchased by ORCL. Overlapping products – portal servers, application servers, service busses, Java development platforms, whatever – will be culled: “synergies” throughout the two organizations will be executed. ORCL will attempt to cross sell its own offerings into the BEAS client base and migrate them away to ORCL alternatives.

Professional investors understand liquidity. Even if they are new to investing in IT equities, they therefore should have little difficulty in understanding that enterprise IT customers likewise yearn for liquidity of technology assets. In the past, enterprise vendors have been slow to offer liquidity. Now, the IT industry is changing fast, and the potential upside for investors in IT is vendors who understand and are executing on technology liquidity: vertically integrated illiquid stacks are from a former and sub-prime era.

Think liquidity.

There’s an interesting interview with Niall McCullough, architect, yesterday in the Irish Times weekend magazine, about the new version of his book “Dublin: An Urban History” (unfortunately the Irish Times online is only premium paid-for content so I can only give you this url to the article). There’s also an interesting web site, giving additional histories of Dublin and the patterns which have shaped it at www.reflectingcity.com.

One of the things which I had not realized about Georgian Dublin is that the buildings, which of course are a part of our heritage, were apparently in general not built to last! The article says: “Based on a land-lease system, the terraces and squares were designed to stand for the lifetime of the lease, usually between 40 and 100 years, whereupon they would be torn down and built again.” Perhaps this is one reason why the Georgian Society has had so many challenges in trying to preserve the best of Georgian Dublin for us and for future generations.

Earlier in the last week, I was in Liaoning province in north east China, for Sli Siar, for discussions relating to the “Five Points, One Line” strategy to re-invigorate one of the old industrial parts of China. As you probably know, Chinese government policy (until recently) is that land is only available for lease, and is owned by the State. The lengths of leases are set by national law, and vary between 40 and 70 years, depending on the land use: residential, commercial, industrial etc. In fact, China has been through various land reform policy changes: the 1946 reform in which land was expropriated from the landlords and equitably distributed to individual households in rural villages; the collectivisation period during the Great Leap Forward in the 1950s in which land was grouped into shared communes; and the current system introduced in the late1970s during Deng Xiaoping’s reforms. I say “the current system”, because in fact just last month, the National People’s Congress passed a new property law, after many years of deliberations, which recognizes the status of private property including land. In my own experience, urban planners and developers today in modern China in general expect their buildings and developments to survive the length of leases of the land on which they are built.

Then on Friday, having traveled back to Europe the previous day, I was with some folks from IONA, visiting one of our customers in the financial services sector, in Zurich. The meeting was with their senior architect – let me call him Tom. Tom is responsible for their group-wide, global IT architecture, and the meeting was to discuss SOA strategy. One of the very interesting points he and I discussed was that there is a sense in the enterprise IT industry, that at long last, we collectively in the enterprise software industry are “building to last”.

As an aside, I do wonder whether “architect” is the appropriate title for somebody like Tom. I tend to think of civil architects as professionals who design buildings. Rather somebody like Tom is really an “urban planner” – he presides over the current and future infrastructure of an entire software city, on which individual applications – buildings – are built.

Anyway. With previous middleware technologies – DCE, DCOM, CORBA, J2EE, etc – and with all due respect to those thousands of technologists world-wide who worked to create these technologies, I think there was always an expectation amongst senior architects (urban planners ?) and IT visionaries, that each of these technologies would fade in time. Sure, each might be strong enough to last for a decade or so, but business logic and applications that were built to exploit any one of these technologies were constructed in the expectation that a more modern, better middleware technology, would emerge within at most a decade. It was perhaps like the relatively short land leases of Georgian Dublin I mentioned above: build your artifacts in the expectation of re-building them a few years later. And so the middleware world proved to be,

Tom postulated that, at long last, enterprise software architects (urban planners ?) can be like the Victorians of over a century ago: laying down infrastructure – whether it be urban water supply, underground and metro train networks, or even sewage pipe networks (what is the best analogy for middleware ? – I leave it to your personal prejudice!) – that will last for a hundred years or so. Is SOA the end of middleware as we know it ? Isn’t SOA good enough to give us a stable infrastructure for at least a hundred years ?

Well, my view is yes, I agree that it is but with one proviso: one has to construct a SOA based (urban-like, city) environment in the expectation that middleware technologies will in fact continue to evolve and change. SOA may mark the end of middleware as we know it, yes Tom, but its chief contribution in this context is its meta-level. In the same way that metadata in a database allows one to reason and manipulate the underlying data, so should a SOA system enable one to reason and manipulate the underlying middleware.

SOA capabilities in frameworks like Artix and its open source companion Celtix allow a de-coupling between business logic and services, and the underlying middleware. In particular, future middleware technologies can be inserted into such frameworks. The meta-level capabilities enable dynamic re-configuration, including for example interface versioning, data versioning, retooling and end-point guardianship.

With SOA, we have indeed reached the end of middleware as we know it, and can now enable enterprise applications which can be built to last.

How do you change an entire nation ?

It’s a topical and fascinating discussion to have over a coffee, and clearly is preoccupying a number of people right now. You may be aware elsewhere from this blog that I have been a frequent visitor to China since 2000: the incredible social shepherding of this enormous nation must be one of human history’s extraordinary moments. I found George Packer’s book on the American administration’s work in Iraq both entirely credible and frightening. My trip to KwaZulu Natal province in South Africa last November with UNICEF brought home to me the frustration of a nation challenged to put its wealth and talent to work in rebuilding after a brutal regime. Closer to home, it is intriguing to watch the recent developments in Northern Ireland as Ian Paisley of the DUP and Gerry Adams of Sinn Fein sit down together to form a devolved government.

Changing an entire enterprise IT infrastructure should not be as demanding as changing an entire nation, but sometimes I’m sure some of us wonder. The introduction of a Service Oriented Architecture, and the management of a Service Oriented Architecture, at enterprise scale, and almost certainly globally, gives justifiable cause for reflection.

As you may have seen if you follow IONA, we announced our first product in the SOA repository and registry sector last week. While some observers expressed strong interest, nevertheless certain others believed that IONA is entering a somewhat crowded space, with various other existing pure play and bundled repository/registry products already out there.

The essential difference, IMHO, between a SOA repository and a SOA registry is in fact largely historical, and most vendors in the space – including now IONA – offer a combination. A SOA registry is (traditionally) used to register WSDL interface definitions and to catalogue available services which implement particular interfaces. It is particularly useful at development time to guide software engineers to discover and re-use existing definitions. A SOA repository is (again, traditionally) a runtime store in which certain meta-data about WSDL interfaces and services, and certain governance policies, are recorded, and made available for interrogation by applications and by the underlying middleware substrate.

In a SOA environment, the distinction between a development time phase and a runtime phase is arguably a little artificial – and as a result, the industry is moving to the combination of repository and registry functions within the same products. A new service is evolved over its lifetime within the SOA, and must both be designed and maintained within the context of other operational services. Certainly, a new service must be deployed with care, adequate testing, and appropriate “sand-boxing” so that it does not disrupt the operations of other services; but equally it cannot be developed completely in isolation to the operational effectiveness of the current system. SOA systems are inherently loosely coupled, and thus full testing of the interdependency between services usually ultimately happens carefully in an operational setting.

The governance of SOA systems is a key theme of the vendors – again, including IONA – in this space. Combined registry and repository products are positioned as assisting the judicious management of both the development and operation of a SOA. No doubt “SOA governance” has been chosen carefully as a catchphrase by the SOA industry, resonating as it does in some business executives’ minds against the backdrop of Sarbanes-Oxley and other recent corporate legislation worldwide.

The pragmatic challenge for any SOA architect concerned about the governance of enterprise systems is of course that governance policy mechanisms are almost certainly already widely fragmented across the entire software system. Some policy mechanisms may already be embedded in application code and business logic. Some are in stored procedures, relating to the probity of particular databases and their data management applications. Some are described by business processes, and orchestrated by business process engines. And now, SOA registry/repository vendors come along and pitch “SOA governance” using their respective products.

In a SOA system, how do you implement policy ? How do you affect changes in policy ?

Let’s return for just a second to the analogy of governing a sovereign state. There is both the integrity of the state, and strategic management of it, to consider. By integrity, I mean both a legal framework by which the state operates, including an appropriate set of laws, perhaps set against the framework of a constitution; and an appropriate enforcement mechanism by which the state and its citizens can reasonably assured that laws are obeyed. By strategic management, I mean the set of policies – laws but also fiscal and other incentives - which the state chooses to adopt to, for example, grow its economy, educate its children and its citizens, and care for the health of its population. However, strategic management cannot occur unless the fundamental integrity of the state is assured; equally, an assurance of integrity alone does not necessarily lead to the long term economic and social success of a nation.

From this analogy, I believe it may be useful to draw a distinction between the integrity of a SOA system, and the business policies and processes chosen to be implemented using it. I believe that integrity should be defined and enforced using a “SOA Guardian” – I deliberately introduce a new term – whilst business policy implementation is rightly the domain of business process engines and orchestration.

Interestingly, business process mechanization and SOA service orchestration has probably been given more attention, priority and emphasis, by the SOA industry at large than SOA integrity issues. BPEL and WS-CDL are probably the best known business process mechanization initiatives, for which a number of process engines exist, including in open source form. Indeed, orchestrating business services defined using standardized interfaces is one of the key selling points of SOA, if not the very essence of SOA for some protagonists. Perhaps business process orchestration and choreography has had more attention than SOA integrity, since there may be a perception amongst SOA practitioners that line of business managers, and corporate executives, can relate most to corporate business process enhancements, rather than the more complex holistic concept of SOA.

But “salus populi suprema est lex”: Cicero said that the ultimate law is the welfare of the people. If the people are unwell, how they can they be governed ? If an enterprise system is impaired, what benefit is business process orchestration ?

“SOA Governance”, if it denotes anything at all, should encompass both SOA integrity and strategic management of business services orchestration: I believe that that is in fact what line of business managers, and corporate executives, expect from the SOA industry.

SOA integrity is a critical prerequisite to leveraging a SOA for business process orchestration. The boundary between the two is naturally dithered to produce effects which are actually not really of substance: business process orchestration can be overloaded to attempt to define and enforce integrity; integrity functions can be strained to implement business policy rules. In the spirit of lean software services, I argue however for a separation of concerns: SOA integrity should be focused as a complete baseline from which business policies and procedures can be easily defined, rapidly deployed, and safely evolved.

A SOA Guardian should be driven by rules relating to the integrity and unimpaired operation of the entire system. It certainly includes security concerns, and the authentication and authorization of access from principals to SOA hosted services, based for example on LDAP directories. However it also includes performance concerns, such as load management and brisk responsiveness. It knows the configuration of every service, with technical details such as the size of the allocated thread pools, binding and addressing information, runtimes and containers across the (almost certainly, highly heterogeneous) SOA deployment: it can be used to re-boot any failed service or even ultimately the entire enterprise. It includes transactional integrity, fault and outage management, so that the temporary loss of one or more SOA hosted services does not cause catastrophic failure. It includes management of change, including management of schema and version changes of services and interfaces. In particular, it includes the planning and execution of technology change, so that old technologies and services can be phased out, rationalized and consolidated, without adversely affecting operations.

It is clear that a SOA Guardian has both – forgive me folks, I’m an engineer by background! – sensors and actuators. That is, it collects and presents operational information relating to the integrity of the system. Equally, it is the mechanism for implementing and enforcing changes to the integral operation of the system. It is clear that the Guardian itself needs to be sound and cohesive: a Guardian system will itself likely be federated and fault-resilient. Because of the heterogeneous nature of most SOA environments, a SOA Guardian may well use combined repository and registry products from other vendors in order to administer details of certain proprietary environments. A SOA Guardian is thus much more than the emerging generation of combined repositories and registries.

Having a repository to record decisions and rules relating to the integrity of the system is certainly useful: the library of a parliament records the laws made therein. Having a console from which operational metrics can be gathered is also important: laws should be made in the context of the society they influence, and it is the role of elected representatives to reflect the views of the populace and the current state of society. But a SOA Guardian also includes the enforcement mechanisms to impose change on a system. Some “SOA Governance” products, surprisingly, apparently are weak or even non-existent in this regard.

For a SOA Guardian to enforce a change, it should clearly be desirable that the entire SOA system not be brought to a halt. Changes in security integrity are already routinely dynamically enforceable in most systems. Changes in configuration to improve performance can sometimes be dynamically implemented in some systems. Changes in fault avoidance and outage management can likewise be dynamically enforced in some systems. Changes in schema and interface versions likewise. Changes in technology are perhaps the most difficult to dynamically implement.

A SOA Guardian, given the correct meshing with the middleware substrate, should nevertheless be able to dynamically deploy all such changes to the integrity of an operational SOA, including changes not hitherto envisaged or previously planned. There should nevertheless be no need to update application business logic or business orchestration in making changes to the integrity of the substrate.

Provisioning changes in telecommunications infrastructures is both a science and an art, based on operational experience. Vendors with experience of such realtime provisioning may emerge to become one source of what I have called SOA Guardians.

How do you change an entire nation ? How do you successfully effect evolution of an entire system ? My view is “carefully”, using small incremental and rapidly evaluated steps, along with a strong separation of concerns. The integrity of a SOA implementation is different from the orchestration of business services to exploit a SOA deployment.

“SOA governance” tools may (perhaps, for some vendors even deliberately) confuse the issue. I encourage the industry to instead separate the concept of SOA guardianship from SOA orchestration. A combined repository/registry tool is a step towards a SOA guardian: dynamic binding, late decision, immediate execution and enterprise wide deployment, is also needed for complete integrity and agility.

In my previous post, I discussed a key challenge facing executives, such as the real VP, K-san, who in principle wish to generate value by adopting SOA across their enterprise. I summarized how Lean Manufacturing and Lean Design gives Toyota the confidence to claim the tagline (here in Ireland at least!): “The best built cars in the World”. I also noted the emergence of Agile thinking in software development thinking.

SOA should impact an entire enterprise’s computing capability as well as business architecture, and needs to be more holistic than specific software development practices such as XP or Scrum. Lean principles can be applied to SOA, whether or not Lean and Agile techniques are also applied to the actual software development of services in SOA. While I personally advocate that software development should be agile and lean, some enterprises may chose not to do so whilst still nevertheless applying Lean thinking to their architecture. The macro principles do not necessarily imply micro ones. The focus of what I call Lean Software Services - LeSS - is empowering people within an organization to focus on their key responsibilities and opportunities, without being overwhelmed by technology choices.

SOA is more than just another technology fad. It is not an issue merely for the CIO: it impacts the CEO and the Board room. It fundamentally impacts the business architecture of an enterprise, as tasks and services undertaken both directly by humans (alone), machines (alone), and humans and machines (acting together), are composed into business choreographies. The composite processes which result are fundamentally different from the classical stove pipes and “silo-ed” applications which are sometimes found as a consequence of history and immediacy: specific business support is needed, specific applications are purchased or built to address these urgent requirements, and once operational these applications become valuable – if not critical – to business operations. Another cause of software silos is merger and acquisition activity, which sometimes result in businesses being urgently combined by operating two software business stacks side by side. It is not that unusual to see operators – for example in a call centre – using multiple physical screens to interact with entirely separate backend systems. SOA is fundamentally and dramatically different: it creates enterprise business value by horizontal impact across the entire business and IT architecture.

SOA embraces a number of software strategies:

· user interfaces, including both traditional screens and new (eg handheld) devices;

· the choreography of software services, and thus the composition of various business logic components to implement business process flows across the enterprise;

· software services, which are the software implementations of those business logic components;

· and finally the underlying substrate of hardware, operating software, databases and middleware.

Separate to each of these four strata, but having influence on all four of them, is the governance of the architecture. Governance decisions arise from both operational and executive concerns: they include for example variation of operational capacity, operational fail-over and backup policies, but also executive led audit trails, authentication policies, and authorization procedures.

In considering Lean principles and SOA, one of the key Lean strategies is the reduction and elimination of waste. Henry Ford introduced “dock to factory floor” inventory avoidance, in which incoming materials were not warehoused but used immediately on the production floor. Toyota subsequently adopted the practice as part of their Just In Time inventory policy. The Poppendiecks interpreted waste elimination in software development as the minimization of unreleased software: code in development, and not yet in production. Partially done work – whether in manufacturing or software development – represents sunk cost but as yet unrealized value.

The Poppendiecks advocate a policy of delaying commitment – postponing writing any software – until there is clear evidence of precisely what is needed. Fuzzy and uncertain requirements lead to wasted effort and probable “bloatware”. Folklore asserts that the Pareto Principle in general applies to software services: 80% of users use only the same set of 20% of the features of a software system, and that by implication 80% of the features are of marginal value. I personally am unaware of any specific research data that supports the folklore, but it’s a good attention grabber for any executive! Kent Beck and his colleagues in the XP world also advocate cautious commitment until customer expectations and “stories” are well understood.

Folklore also has it that the cost of changing a software program increases (exponentially) with time. Delaying commitment therefore appears to destroy value. Lean Design and Agile/XP on the contrary increase value by delaying commitment, benefiting from modern software technologies which have reduced the costs of change.

Consider a fully flexible airline ticket, allowing you to select any flight over the next month. It is more valuable, and costs more, than a fixed ticket for a specific flight. Think also of stock options: an option to buy or sell something in the future has value right now. Delaying commitment can create value, not destroy it: see the excellent paper by Hakan Erdogmus and John Favaro who apply financial option analysis – including Black-Scholes option valuation - to software development using XP.

Naturally, if you want full flexibility for an airline ticket, you go buy yourself an aircraft of your own. The longer the time you have before an option to do something (fly, buy a stock, develop a software program) lapses, the more valuable the option is: I guess that’s one reason private aircraft are so expensive.

Of course, in the same way that delaying a decision can improve the discounted cash flow (DCF) value of a software implementation, poor subsequent execution can destroy the value by delaying positive cash flow. The Poppendiecks conjunct delayed commitment with rapid execution: once a decision is actually (eventually) made, it is put into practice extremely quickly. It is no good having an expensive private aircraft if it is out of service when you eventually decide to use it.

Seems a bit of a digression from SOA, perhaps ? Bear with me. As I noted above, the focus of LeSS is empowering people in the organization to work on their key responsibilities and opportunities, without being overwhelmed by technology choices. The highest value comes from delaying a decision to the opportune time, and then executing immediately. And then, perhaps later on, making a new decision, and executing immediately.

In fact, the ultimate value results if the decision can be orthogonal to the environment it affects. That is, the decision can be taken at any time, and re-considered at any further time, without restriction from the environment. Your private aircraft should be available any time you want it. To quote from Wikipedia: “Orthogonality guarantees that modifying the technical effect produced by a component of a system neither creates nor propagates side effects to other components of the system. The emergent behaviour of a system consisting of components should be controlled strictly by formal definitions of its logic and not by side effects resulting from poor integration, i.e. non-orthogonal design of modules and interfaces. Orthogonality reduces testing and development time because it is easier to verify designs that neither cause side effects nor depend on them.” A classic example from hardware design is that any computer instruction can be applied to any memory location, without restriction.

A good architecture – particularly a SOA – meets or exceeds expectations for run-time stability and performance, while minimizing the investment – time, financial, and human – to create, modify and maintain it. Orthogonality is an excellent strategy.

As we reflect on the four SOA layers I mentioned above (user interface, choreography, software services, and substrate) together with governance, LeSS insists that all are mutually orthogonal, yielding a separation of concerns. If this is achieved, staff are empowered to focus on their key responsibilities and opportunities, without being overwhelmed by technology choices. Learning can be amplified without corruption or qualification arising from other layers. Decisions can be delayed until appropriate information is known for a specific activity – whether designing a new screen, building a new choreography routine, or implementing a new software service – and then executing quickly, without recourse to factors arising in orthogonal layers. Integrity can be strengthened by building testing, assertions and governance without risk of stress cracks being induced from adjacent layers. In summary, the principles of Lean Design as advocated by the Poppendiecks and others, can be applied to SOA.

It goes further. My VP friend, K-san, noted in my previous blog posting of his concern of the lack of a talent pool for SOA. In particular, what technologies and industry standards do software developers need to implement SOA components ? What skills should I train my staff for, and what skills should I hire against ? Using LeSS, the answer is that software developers coding business logic as software services can pretty much use whatever technology with which they are comfortable – C#, Java, C++, C, Ruby, even Cobol; along with, as appropriate to the selected technology, message definitions, copybooks or IDL. The choreography of the software services which they implement, and the infrastructure to support them, should be – and are, is a LeSS environment - orthogonal concerns.

These are bold claims. Can the SOA strata be orthogonal ?

Let us start on the user interface and choreography layers. The LeSS claim is that user interface specialists and choreographers should independently design and implement. User interface specialists focus on the ergonomics of screen layouts, devices and the human-machine interface. Their primary attention is ensuring efficient dialogue with the software infrastructure, and minimizing misunderstanding resulting from inadequate layouts, poorly designed graphics and inconsistent command activation. Choreographers are concerned with business process flows across a set of collaborating business logic components, implemented as software services. The routines which they design ensure that business procedures are safely implemented and, further, that these can be rapidly extended and modified as business needs dictate. The routines in turn are implemented by a business process engine, of which several alternatives exist – both open source and vendor produced. In practice, most business process engines, certainly of which I am aware, do not impact very much on the user interfaces offered by the software services which they choreograph. Orthogonality at this particular boundary does not appear an issue.

Business logic components implemented as software services frequently require user input and guidance, and present information. In SOA, can the user interface and software services strata also be orthogonal ? In practice, user interfaces designed for browser access today separate “look’n’feel” from actual dialogue, by using for example CSS. Indeed there is a trend to further separation of business logic implementation in SOA, from presentation and dialogue to end users, by new client side tools based on Ajax technology – for example “html scraping”, data mashups and personalized portals. These browser based tools are orthogonal to the enterprise software services with which they interact.

Let us move on. Choreography tools, such as business process engines, ensure appropriate process flows between specific software services implementing business logic. Each software service implements a prescribed interface encapsulating its functionality – indeed this is a primary motivation for SOA, as I noted in my previous post. Thus, so as to control the process flows across software services, a business process engine has to understand the interface definitions. This begins to get at the root cause of my VP friend K-san’s issue: what technology should be used for interface definitions ? Can the choreography and software service strata really be orthogonal ?

A similar issue arises at the boundary between the software services and the underlying enterprise substrate (of middleware, databases, operating systems and hardware): can these two strata also be orthogonal ? Programming language portability used to be a major issue a couple of decades ago: it arguably is not today, due to dynamic languages (such as Ruby and PHP), bytecode interpreters (eg for Java), and platform virtualization technologies (such as VMware and Xen). On the other hand, middleware substrates and software application programming are usually strongly mutually coupled, and are hardly ever orthogonal. Business logic is written to exploit a specific middleware technology, whether it be message definitions for MQ, Tibco Rendezvous or JMS; or FML for Tuxedo; or IDL for CORBA; copybooks for Cobol; or J2EE EJBs or .Net components and WSDL.

LeSS strongly argues that software developers writing business logic to implement software services should be able to do so regardless of specific middleware technologies. Of course, each software project does have to make some choice: a C# developer selects Windows and .Net for her substrate; a mainframe programmer selected Cobol, copybooks and CICS; and a Java developer perhaps 10 years later choses Linux and JMS. Different components may be implemented in different technologies, for reasons relating to history and/or skills availability. It is the role of the middleware substrate to allow such decisions to be independently made, concurrently and also across time.

Please note that I am not arguing for complete transparency of the distributed infrastructure to the software developer. I can recall certain research projects, and certain commercial products back in the early 90s – lets not name them for fear of embarrassment – advocating that any (fine-grained) software interface should be capable of remote invocation, so that a software program can be provisioned arbitrarily across a network of machines! Rather, LeSS asserts that each software service should explicitly identify (at least) one interface as available for use by other software services (regardless of their technology), and quite probably from a remote machine. However the technology – MQ message definitions, CORBA IDL, WSDL, whatever – chosen to do so, may vary from software service to software service.

Please note too that I am not advocating universal availability of all technologies across the entire substrate of SOA: for example, it does not make sense to make MQ and CORBA and JMS and Tuxedo available everywhere across the enterprise. Instead, using Lean principles, a specific technology should be made available only when and specifically where needed: just in time. In particular, making a specific technology available at the core of the system – for example in a central hub – may be a prime target for the (Lean principle of) eliminating waste. When software is available, but infrequently used, it may be an example of sunk cost and poorly realized value. LeSS advocates provisioning and installation of substrate software – e.g. a messaging system like JMS – only at those specific software services (end-points) which need it at this time.

What of the governance of a SOA system ? Lean Manufacturing systems pay extraordinary attention to ensure smooth flow across the factory infrastructure even in the face of fluctuating demand and micro-orders from customers. Resource hubs are in general a source of challenge and problems: de-centralisation and dynamic allocation are fruitful tactics to minimize contention and to anticipate demand waves. LeSS likewise emphasizes that software business logic should be orthogonal from dynamic provisioning of capacity and capability. In general, dependence on central software hubs (usually on dedicated servers) is a poor tactic, even more so if a source of sunk cost and poorly realized value.

Drivers of governance and substrate changes can be very many: for example, security policy improvements; fail-over enhancements; change in choice of messaging protocol so as to reduce cost; change in a data format so as to modernize and improve integration; and re-factoring of a service interface definition schema so as to improve re-use, add clarity and enhance development agility. Versioning of service interfaces is common in the evolution of practical SOA implementations. Such policy enhancements in the governance, and improvements to operation of the enterprise substrate, should not require human intervention in the business software.

My friend K-san was concerned about the lack of skills for SOA: the technology choices available overwhelm a clear selection across the enterprise, leading to risk in staff selection and skill sets. LeSS empowers staff in the organization to focus on their key responsibilities and opportunities, without being overwhelmed by technology choices. Each project team can chose to use the technology with which they are most familiar. Separation of concerns, and orthogonality, can be achieved – today - by appropriate selection of off the shelf products.

Lean principles are being applied by many software development teams worldwide to eliminate waste, delay decision making until facts emerge, execute fast, make integrity inherent, amplify learning, empower teams and drive a holistic view of their system.

Lean principles can also be applied at the macro-level, to entire enterprise architectures. Learning can be amplified without corruption or qualification arising from other layers. Decisions can be delayed until appropriate information is known for a specific activity – whether designing a new screen, building a new choreography routine, or implementing a new software service – and then executing quickly, without recourse to factors arising in orthogonal layers. Integrity can be strengthened by building testing, assertions and governance without risk of stress cracks being induced from adjacent layers. The principles of Lean Design as advocated by the Poppendiecks and others, can be applied to SOA.

Separation of concerns, supported by technology which enables orthogonal decision making, and just in time commitment, is the key to Lean Software Services. In adopting SOA, ensure your chosen supplier or vendors provide you with orthogonal, just in time choices throughout the entire architecture. Then empower your teams to use technology with which they are familiar.