chris horn's blog

Intel confirmed yesterday its 2.5B$ commitment to build Fab68 in Dalian, China Apparently it is the first green field development since 1992 when Intel came to Leixlip Ireland. Dalian is one of China’s greener cities, and set in Liaoning Province near the border with North Korea.

By complete coincidence, there is a large delegation of Chinese software companies and associated Government officials visiting Ireland at the moment, with many of them from Dalian. I attended a meeting yesterday morning hosted by the Department of Enterprise, Trade and Employment, along with the IDA, Enterprise Ireland and the Irish Software Association for the visiting delegation.

Listening to the alternate presentations – Irish and Chinese – it struck me how different our two software industries are at this time.

The Irish software industry focuses on the global export market, because the home market is so small. From the figures presented this morning, most Chinese software companies currently focus on their domestic market, and do not export very much at this time by comparison.

The Irish software industry has six times as much revenue generated by multinationals operating in Ireland as by indigenous Irish companies. The Chinese numbers are the converse: more indigenous activity than multinational at this time.

The Irish software industry is primarily focused on the creation of new software products, and associated services. The Chinese industry at this time is focused instead on software outsourcing and business process outsourcing.

One can understand the Chinese focus, given the need to create employment, particularly in the private sector. At the same time, it was interesting to hear an official from Dalian readily admit to staff shortages, leading to upwards cost pressures: in fact the local government is apparently offering housing subsistence allowances and tax breaks to software professionals, so as to keep labour costs down in the area. It would be wonderful if our own Government in Ireland took such an enlightened view!

One wonders how these positions will change over, say, the next five years. The Irish industry may need to become less dependent on foreign direct investment and more focused on its indigenous companies. The Irish Software Association is strongly lobbying the Irish Government to considerably increase its IT procurement from indigenous Irish software companies: it is very ironic that many Irish companies in general have much more success selling to Government agencies outside of Ireland than within it. Maybe the Irish domestic market can be grown for the indigenous companies.

The Chinese focus on outsourcing and BPO is also interesting. We were told this morning that in the same way that Bangalore is a centre for outsourcing and BPO for the US and the English speaking world, Dalian is as successful as a centre for outsourcing and BPO from both Japan and South Korea. However, I suspect the Irish appetite in general for outsourcing is more focused on central Europe, driven by the expansion of the European Union last year and the availability of low cost air connections directly out of Ireland to a large number of central European destinations.

Some Irish agency officials pitched Ireland as an opportunity for investment by Chinese companies as a gateway to Europe. In return, some Dalian officials pitched Dalian as an opportunity for investment by Irish companies as a gateway to both Japan and both Koreas. I think this reciprocal perspective holds a mirror to both arguments: frankly I suspect most Irish companies will chose to invest directly in Japan or a Korea and choose to bypass Dalian to do so; I suspect most Chinese companies may think the same in regard to the rest of Europe and using Ireland as a gateway.

A little bit of Yin and Yang this morning. The true opportunity for collaboration comes from a closer working relationship, and working together to deliver whole products jointly together to specific market niches and opportunities. There are many commercial opportunities in China, and my own experience shows that a visible commitment on the ground – for example by opening a development centre – reassures Chinese customers and prospects that a company is in it for the long haul.

This particular posting has very little to do with “the software industry” and more to do with the “other things” , so skip it if you are just interested in technology!

As I mentioned in my first posting, I am (non exec.) Chair of UNICEF Ireland. If you travel transatlantic with Aer Lingus, you may be aware of our “Change for Good” campaign which the cabin staff and airline management promote for us..

I first became involved in UNICEF as a result of an early decision at IONA to designate it as our corporate charity. I joined the UNICEF Board level back in 2001 after IONA underwrote a major auction for HIV/AIDS at Sothebys in New York for memorabilia donated by the movie industry, as a result of a wonderful initiative taken by Liam Neeson.

Maura Quinn got me involved, in her role as the Executive Director. Maura has been a tireless leader and advocate for children at UNICEF, since starting in 1996. She has been a frequent visitor to UNICEF’s field operations worldwide. She put the “Movie action for kids” initiative together, and was the driver behind the UNICEF world wide initiative on HIV/AIDS – a decision taken when we hosted the annual global conference of the national fund-raising committees in Dublin in 2004. During her tenure, the Irish Government has dramatically increased its giving to the third world and is well on its way to its public commitment to invest 0.7% of GDP, and in particular – and I believe as a direct result of Maura’s intervention – for children and HIV/AIDS.

Maura came to me last September and told me she had decided, after ten years, to step down as Executive Director. She is doing so for personal reasons, and will be returning to her home town of Roscommon. I am very sorry to see her go, wish her the very best, and am sure that she will continue to be a great friend of UNICEF.

Amrop-Hever very kindly offered their services in finding a replacement for Maura. We had a large number of candidates, and some very strong ones indeed. I am very happy to confirm today that Melanie Verwoerd is taking over from Maura, from April (ie next week).

Melanie is a South African national, living in Dublin. She lived and studied in the UK for a while, but returned to South Africa in 1990 when Nelson Mandela was released from prison. She advocated strongly against apartheid, and on behalf of women and children.

She was invited by Mr. Mandela to stand for Parliament on behalf of the ANC, and was duly elected in 1994 as the youngest ever woman to serve the South African Parliament, and subsequently also re-elected.

In 2001 she was appointed as the South African Ambassador to Ireland. She was involved in the Northern Ireland peace process, and during her tenure both trade and tourism in South Africa from Ireland greatly increased.

In 2005 she finished her post as Ambassador and has remained in Ireland. She has been the weekly presenter on national radio on issues relating to immigrants and children in Ireland. She also has represented the Mandela Rhodes Foundation here.

As far as I know, I believe that this is the first time a major Irish based NGO has appointed a foreign national as its Chief Executive, and I believe that Melanie will bring a fresh perspective and linkage between childrens issues in the developing world and in Ireland.

I enjoyed very much working with Maura. I believe that working with Melanie will be as exciting, and I expect she will be a very powerful advocate for children.

In my previous post, I discussed a key challenge facing executives, such as the real VP, K-san, who in principle wish to generate value by adopting SOA across their enterprise. I summarized how Lean Manufacturing and Lean Design gives Toyota the confidence to claim the tagline (here in Ireland at least!): “The best built cars in the World”. I also noted the emergence of Agile thinking in software development thinking.

SOA should impact an entire enterprise’s computing capability as well as business architecture, and needs to be more holistic than specific software development practices such as XP or Scrum. Lean principles can be applied to SOA, whether or not Lean and Agile techniques are also applied to the actual software development of services in SOA. While I personally advocate that software development should be agile and lean, some enterprises may chose not to do so whilst still nevertheless applying Lean thinking to their architecture. The macro principles do not necessarily imply micro ones. The focus of what I call Lean Software Services - LeSS - is empowering people within an organization to focus on their key responsibilities and opportunities, without being overwhelmed by technology choices.

SOA is more than just another technology fad. It is not an issue merely for the CIO: it impacts the CEO and the Board room. It fundamentally impacts the business architecture of an enterprise, as tasks and services undertaken both directly by humans (alone), machines (alone), and humans and machines (acting together), are composed into business choreographies. The composite processes which result are fundamentally different from the classical stove pipes and “silo-ed” applications which are sometimes found as a consequence of history and immediacy: specific business support is needed, specific applications are purchased or built to address these urgent requirements, and once operational these applications become valuable – if not critical – to business operations. Another cause of software silos is merger and acquisition activity, which sometimes result in businesses being urgently combined by operating two software business stacks side by side. It is not that unusual to see operators – for example in a call centre – using multiple physical screens to interact with entirely separate backend systems. SOA is fundamentally and dramatically different: it creates enterprise business value by horizontal impact across the entire business and IT architecture.

SOA embraces a number of software strategies:

· user interfaces, including both traditional screens and new (eg handheld) devices;

· the choreography of software services, and thus the composition of various business logic components to implement business process flows across the enterprise;

· software services, which are the software implementations of those business logic components;

· and finally the underlying substrate of hardware, operating software, databases and middleware.

Separate to each of these four strata, but having influence on all four of them, is the governance of the architecture. Governance decisions arise from both operational and executive concerns: they include for example variation of operational capacity, operational fail-over and backup policies, but also executive led audit trails, authentication policies, and authorization procedures.

In considering Lean principles and SOA, one of the key Lean strategies is the reduction and elimination of waste. Henry Ford introduced “dock to factory floor” inventory avoidance, in which incoming materials were not warehoused but used immediately on the production floor. Toyota subsequently adopted the practice as part of their Just In Time inventory policy. The Poppendiecks interpreted waste elimination in software development as the minimization of unreleased software: code in development, and not yet in production. Partially done work – whether in manufacturing or software development – represents sunk cost but as yet unrealized value.

The Poppendiecks advocate a policy of delaying commitment – postponing writing any software – until there is clear evidence of precisely what is needed. Fuzzy and uncertain requirements lead to wasted effort and probable “bloatware”. Folklore asserts that the Pareto Principle in general applies to software services: 80% of users use only the same set of 20% of the features of a software system, and that by implication 80% of the features are of marginal value. I personally am unaware of any specific research data that supports the folklore, but it’s a good attention grabber for any executive! Kent Beck and his colleagues in the XP world also advocate cautious commitment until customer expectations and “stories” are well understood.

Folklore also has it that the cost of changing a software program increases (exponentially) with time. Delaying commitment therefore appears to destroy value. Lean Design and Agile/XP on the contrary increase value by delaying commitment, benefiting from modern software technologies which have reduced the costs of change.

Consider a fully flexible airline ticket, allowing you to select any flight over the next month. It is more valuable, and costs more, than a fixed ticket for a specific flight. Think also of stock options: an option to buy or sell something in the future has value right now. Delaying commitment can create value, not destroy it: see the excellent paper by Hakan Erdogmus and John Favaro who apply financial option analysis – including Black-Scholes option valuation - to software development using XP.

Naturally, if you want full flexibility for an airline ticket, you go buy yourself an aircraft of your own. The longer the time you have before an option to do something (fly, buy a stock, develop a software program) lapses, the more valuable the option is: I guess that’s one reason private aircraft are so expensive.

Of course, in the same way that delaying a decision can improve the discounted cash flow (DCF) value of a software implementation, poor subsequent execution can destroy the value by delaying positive cash flow. The Poppendiecks conjunct delayed commitment with rapid execution: once a decision is actually (eventually) made, it is put into practice extremely quickly. It is no good having an expensive private aircraft if it is out of service when you eventually decide to use it.

Seems a bit of a digression from SOA, perhaps ? Bear with me. As I noted above, the focus of LeSS is empowering people in the organization to work on their key responsibilities and opportunities, without being overwhelmed by technology choices. The highest value comes from delaying a decision to the opportune time, and then executing immediately. And then, perhaps later on, making a new decision, and executing immediately.

In fact, the ultimate value results if the decision can be orthogonal to the environment it affects. That is, the decision can be taken at any time, and re-considered at any further time, without restriction from the environment. Your private aircraft should be available any time you want it. To quote from Wikipedia: “Orthogonality guarantees that modifying the technical effect produced by a component of a system neither creates nor propagates side effects to other components of the system. The emergent behaviour of a system consisting of components should be controlled strictly by formal definitions of its logic and not by side effects resulting from poor integration, i.e. non-orthogonal design of modules and interfaces. Orthogonality reduces testing and development time because it is easier to verify designs that neither cause side effects nor depend on them.” A classic example from hardware design is that any computer instruction can be applied to any memory location, without restriction.

A good architecture – particularly a SOA – meets or exceeds expectations for run-time stability and performance, while minimizing the investment – time, financial, and human – to create, modify and maintain it. Orthogonality is an excellent strategy.

As we reflect on the four SOA layers I mentioned above (user interface, choreography, software services, and substrate) together with governance, LeSS insists that all are mutually orthogonal, yielding a separation of concerns. If this is achieved, staff are empowered to focus on their key responsibilities and opportunities, without being overwhelmed by technology choices. Learning can be amplified without corruption or qualification arising from other layers. Decisions can be delayed until appropriate information is known for a specific activity – whether designing a new screen, building a new choreography routine, or implementing a new software service – and then executing quickly, without recourse to factors arising in orthogonal layers. Integrity can be strengthened by building testing, assertions and governance without risk of stress cracks being induced from adjacent layers. In summary, the principles of Lean Design as advocated by the Poppendiecks and others, can be applied to SOA.

It goes further. My VP friend, K-san, noted in my previous blog posting of his concern of the lack of a talent pool for SOA. In particular, what technologies and industry standards do software developers need to implement SOA components ? What skills should I train my staff for, and what skills should I hire against ? Using LeSS, the answer is that software developers coding business logic as software services can pretty much use whatever technology with which they are comfortable – C#, Java, C++, C, Ruby, even Cobol; along with, as appropriate to the selected technology, message definitions, copybooks or IDL. The choreography of the software services which they implement, and the infrastructure to support them, should be – and are, is a LeSS environment - orthogonal concerns.

These are bold claims. Can the SOA strata be orthogonal ?

Let us start on the user interface and choreography layers. The LeSS claim is that user interface specialists and choreographers should independently design and implement. User interface specialists focus on the ergonomics of screen layouts, devices and the human-machine interface. Their primary attention is ensuring efficient dialogue with the software infrastructure, and minimizing misunderstanding resulting from inadequate layouts, poorly designed graphics and inconsistent command activation. Choreographers are concerned with business process flows across a set of collaborating business logic components, implemented as software services. The routines which they design ensure that business procedures are safely implemented and, further, that these can be rapidly extended and modified as business needs dictate. The routines in turn are implemented by a business process engine, of which several alternatives exist – both open source and vendor produced. In practice, most business process engines, certainly of which I am aware, do not impact very much on the user interfaces offered by the software services which they choreograph. Orthogonality at this particular boundary does not appear an issue.

Business logic components implemented as software services frequently require user input and guidance, and present information. In SOA, can the user interface and software services strata also be orthogonal ? In practice, user interfaces designed for browser access today separate “look’n’feel” from actual dialogue, by using for example CSS. Indeed there is a trend to further separation of business logic implementation in SOA, from presentation and dialogue to end users, by new client side tools based on Ajax technology – for example “html scraping”, data mashups and personalized portals. These browser based tools are orthogonal to the enterprise software services with which they interact.

Let us move on. Choreography tools, such as business process engines, ensure appropriate process flows between specific software services implementing business logic. Each software service implements a prescribed interface encapsulating its functionality – indeed this is a primary motivation for SOA, as I noted in my previous post. Thus, so as to control the process flows across software services, a business process engine has to understand the interface definitions. This begins to get at the root cause of my VP friend K-san’s issue: what technology should be used for interface definitions ? Can the choreography and software service strata really be orthogonal ?

A similar issue arises at the boundary between the software services and the underlying enterprise substrate (of middleware, databases, operating systems and hardware): can these two strata also be orthogonal ? Programming language portability used to be a major issue a couple of decades ago: it arguably is not today, due to dynamic languages (such as Ruby and PHP), bytecode interpreters (eg for Java), and platform virtualization technologies (such as VMware and Xen). On the other hand, middleware substrates and software application programming are usually strongly mutually coupled, and are hardly ever orthogonal. Business logic is written to exploit a specific middleware technology, whether it be message definitions for MQ, Tibco Rendezvous or JMS; or FML for Tuxedo; or IDL for CORBA; copybooks for Cobol; or J2EE EJBs or .Net components and WSDL.

LeSS strongly argues that software developers writing business logic to implement software services should be able to do so regardless of specific middleware technologies. Of course, each software project does have to make some choice: a C# developer selects Windows and .Net for her substrate; a mainframe programmer selected Cobol, copybooks and CICS; and a Java developer perhaps 10 years later choses Linux and JMS. Different components may be implemented in different technologies, for reasons relating to history and/or skills availability. It is the role of the middleware substrate to allow such decisions to be independently made, concurrently and also across time.

Please note that I am not arguing for complete transparency of the distributed infrastructure to the software developer. I can recall certain research projects, and certain commercial products back in the early 90s – lets not name them for fear of embarrassment – advocating that any (fine-grained) software interface should be capable of remote invocation, so that a software program can be provisioned arbitrarily across a network of machines! Rather, LeSS asserts that each software service should explicitly identify (at least) one interface as available for use by other software services (regardless of their technology), and quite probably from a remote machine. However the technology – MQ message definitions, CORBA IDL, WSDL, whatever – chosen to do so, may vary from software service to software service.

Please note too that I am not advocating universal availability of all technologies across the entire substrate of SOA: for example, it does not make sense to make MQ and CORBA and JMS and Tuxedo available everywhere across the enterprise. Instead, using Lean principles, a specific technology should be made available only when and specifically where needed: just in time. In particular, making a specific technology available at the core of the system – for example in a central hub – may be a prime target for the (Lean principle of) eliminating waste. When software is available, but infrequently used, it may be an example of sunk cost and poorly realized value. LeSS advocates provisioning and installation of substrate software – e.g. a messaging system like JMS – only at those specific software services (end-points) which need it at this time.

What of the governance of a SOA system ? Lean Manufacturing systems pay extraordinary attention to ensure smooth flow across the factory infrastructure even in the face of fluctuating demand and micro-orders from customers. Resource hubs are in general a source of challenge and problems: de-centralisation and dynamic allocation are fruitful tactics to minimize contention and to anticipate demand waves. LeSS likewise emphasizes that software business logic should be orthogonal from dynamic provisioning of capacity and capability. In general, dependence on central software hubs (usually on dedicated servers) is a poor tactic, even more so if a source of sunk cost and poorly realized value.

Drivers of governance and substrate changes can be very many: for example, security policy improvements; fail-over enhancements; change in choice of messaging protocol so as to reduce cost; change in a data format so as to modernize and improve integration; and re-factoring of a service interface definition schema so as to improve re-use, add clarity and enhance development agility. Versioning of service interfaces is common in the evolution of practical SOA implementations. Such policy enhancements in the governance, and improvements to operation of the enterprise substrate, should not require human intervention in the business software.

My friend K-san was concerned about the lack of skills for SOA: the technology choices available overwhelm a clear selection across the enterprise, leading to risk in staff selection and skill sets. LeSS empowers staff in the organization to focus on their key responsibilities and opportunities, without being overwhelmed by technology choices. Each project team can chose to use the technology with which they are most familiar. Separation of concerns, and orthogonality, can be achieved – today - by appropriate selection of off the shelf products.

Lean principles are being applied by many software development teams worldwide to eliminate waste, delay decision making until facts emerge, execute fast, make integrity inherent, amplify learning, empower teams and drive a holistic view of their system.

Lean principles can also be applied at the macro-level, to entire enterprise architectures. Learning can be amplified without corruption or qualification arising from other layers. Decisions can be delayed until appropriate information is known for a specific activity – whether designing a new screen, building a new choreography routine, or implementing a new software service – and then executing quickly, without recourse to factors arising in orthogonal layers. Integrity can be strengthened by building testing, assertions and governance without risk of stress cracks being induced from adjacent layers. The principles of Lean Design as advocated by the Poppendiecks and others, can be applied to SOA.

Separation of concerns, supported by technology which enables orthogonal decision making, and just in time commitment, is the key to Lean Software Services. In adopting SOA, ensure your chosen supplier or vendors provide you with orthogonal, just in time choices throughout the entire architecture. Then empower your teams to use technology with which they are familiar.

A VP friend at major global corporation – let me call him K-san, rather than give his complete name and afflliation - put it to me thus last week: “The fundamental issue that we have with SOA is skills shortage. With CORBA, it was much easier: there was just one, relatively cohesive, family of standards, and we could recruit to these. In the good days of J2EE, equally so. Now today, SOA is just too ill-defined, and a common set of skills are just not there in the market”.

Wow. Despite all the excitement, all the momentum, all the vendor investment in Service Oriented Architectures (SOA), we in the enterprise software industry are failing to produce an adequate talent pool for our customers. SOA is an architectural style in which complex enterprise systems are composed from business logic components, structured as loosely coupled software services. Because SOA is an architectural style, rather than a specific technology, some (many ?) apparently find it difficult to make a commitment to SOA, and to find staff.

Why is SOA relevant ? One frequently cited justification is – at long last – a common basis for dialogue between business executives and IT specialists. The fundamental building block for SOA is a specific business function, usually encapsulated as a software service. A business function may also be a step purely carried out by humans, with no direct machine intervention. Business processes can be choreographed as appropriate sets of interacting (business level) software services, and human steps if necessary.

A key business value which results from SOA is encapsulating, or wrapping, each business logic component as a (usually, software) service. Other parts of the system can then make use of each such service, regardless of the physical location (in-house or off-premises) or specific technology used to implement that particular service. In turn, a business judgment that a service could be profitably re-located or re-implemented can be executed upon, without a cascade of changes being required on those other components which use that service.

A second, perhaps less obvious, business value is the reduction in effort to implement new business services. This can arise both because business components can be choreographed to work together in concert, and new routines can then be quickly defined to provide new services; and also because new business components can be developed and added into the portfolio of available software services, re-using where appropriate some of the functionality already available elsewhere in the portfolio.

From my own personal experience, the epitome of a practical enterprise scale SOA initiative is that undertaken by Credit Suisse First Boston, built on products from IONA and other vendors, starting in the late 1990s under the leadership of Stefan Murer and Martin Prater. I am sure there are others. The CSFB experience is well summarized in online presentations by Hermann Schlamann and Keith Tice. In some of their SOA metrics – see the presentations - CSFB state they have over 900 business logic components implemented as software services, used by over 150 composite enterprise applications, with an average of 3.7 services used per client application, 43% re-use and development savings of over 12,000 mandays. I understand that more recent figures show even high re-use figures and savings.

But back to my VP friend K-san. In the good old days, you could use CORBA to do SOA – and CSFB did do so. Today, there is confusion about SOA technologies and standards, yielding a perceived show stopper: lack of skilled staff.

I take it as a given that staff are professional: everybody does the best job they can, given their education, training, abilities and knowledge. My own view is that it is hardly the fault of the people: it is the fault of the process. What is it about our approach to SOA that makes it difficult to find people ? How can we change our thinking about SOA so that it becomes tractable, pragmatic, and harder for corporations to make expensive mistakes ?

In chatting to Larry Alston about this, he reminded me of the “people to process” transition made by Henry Ford. Henry Ford did not invent the automobile: the problem was that up until Henry arrived on the scene, cars were being hand-crafted by extraordinarily skilled, and expensive, workers. Henry Ford focused on process: he empowered ordinary mortals to reliably build cars by creating the correct processes, and paid them high wages to do so.

Today’s manufacturing processes are derivative of the innovation of Lean Manufacturing. As K-san effortlessly reminds me, Lean is of course associated with Toyota, and specifically the founder Sakichi Toyoda, his son Kiichiro, the external consultant Shigeo Shingo; the engineer Ohno Taiichi, and the HR executive Isao Kato. The story goes that, in the 1950s, a delegation from Toyota visited the Ford manufacturing plants in Michigan. Although Ford was a global industry leader at the time, the Toyota people were apparently not particularly impressed by what they saw at Ford. In particular they observed large quantities of inventory sitting idle, and noted that the production across the entire plant did not flow smoothly, with staff occasionally just waiting around.

By contrast, during their visit they happened to be shopping in a local supermarket, and observed the simple idea of an automatic drink re-supplier: once a customer takes a drink, another is automatically ordered and replaces it. The supermarket deferred re-ordering and re-stocking goods until customers made purchases.

On return to Japan, Toyota further developed the basic concepts of what is now known as the Toyota Production System, which greatly reduced lead-time and wastage, whilst simultaneously improving quality. The basic ideas of “Lean Manufacturing” were subsequently taken into Toyota’s R&D and design shops, yielding “Lean Design”. Today, Toyota use the tagline ( at least here in Ireland, where we have no car manufacturing of our own and thus import global brands): “The best built cars in the World.”

How do we turn ordinary mere mortals in the software industry into powerfully productive professionals for SOA ? Is it just a matter of education, training, abilities and knowledge ? What about process ? What can we in the software industry learn from what has gone before ?

Lean thinking has had a dramatic impact on the global software community. Significant momentum in Lean Software Development and Lean Product Development developed back in 2003 when Mary and Tom Poppendieck published their excellent book on the topic. The Poppendieck’s studied the lessons of Lean Manufacturing, and particularly Lean Design, and applied it to software development. Elimination of waste, delayed decision making, fast execution, integral integrity, learning amplification, team empowerment are the key factors which they identified in driving a holistic view of the process.

Agile software development is related, but had a different starting point. Rather than a heritage in manufacturing systems, Agile has its nemesis in software development projects and by developers themselves. It is particularly associated with Kent Beck, Ron Jeffries, and Ward Cunningham (now at eclipse.org and a committer to the Eclipse Spaces project which I mentioned last week), during their work on development of a payroll system for Chrysler in 1996, leading to the Extreme Programming (XP) movement. However Wikipedia also notes that the "practice of test-first development, planning and writing tests before each micro-increment" was used as far back as NASA's Project Mercury, in the early 1960s.

Thad Sheer provides an excellent explanation of the differences in philosophy between Lean and Agile approaches in his 2005 blog post. Lean projects need not necessarily be Agile, and vice versa. In summary, he suggests that Lean usually gets more initial corporate support than Agile since it uses concepts to which most executives can relate, rather than terminology more appropriate to software developers. He also states: “There’s a lot more bang for the buck vis-à-vis the bottom line with a partially completed Lean initiative than a partially completed Agile initiative. Once an organization is LEAN, becoming AGILE is a step that developers can take on their own without requiring much management backing.”

In IONA, we started looking seriously at Extreme Programming back in about 2002, when we hosted Kent Beck in person several times to meet with our software development organization. I personally recall that Ciaran Dynes drove much of the initiative within IONA, as one of our engineering managers, and I am grateful for his leadership. Ciaran went on to build our Beijing based development centre – using XP - and is currently our Artix product manager, based out of Dublin.

So Chris, what about SOA, and do Lean and/or Agile apply ? Well, interestingly there is an analysis published by IBM which in its first part summarises various approaches to both Lean and Agile software development; and in its second part discusses that these approaches are not incompatible to SOA. The take-away is “Oil and Water do mix!”

Ooof. Personally, I find such a take-away divisive: it implies that the world of SOA and the world of Lean/Agile are fundamentally confrontational, but that perhaps with sufficient management effort they can probably be brought together. To be fair, the folks at IBM say they were responding to a previous article suggesting that SOA and Lean/Agile actually are incompatible!. All of this is hardly comforting to corporate strategists thinking about SOA adoption, and I suspect would not be greeted with relief by my VP friend K-san.

I think I have a differing view. I believe that a successful SOA results from Lean thinking. I discuss this in my next posting.

Part of the dilemma with the global software community today is that there are so so many open source repositories and projects. I guess amongst the best known community owned initiatives are Apache, eclipse and SourceForge. But there are very many others, as a quick Google will reveal: for example HotScripts, NetLib, RubyForge, and CodeHaus. Most of course are primarily specific to a particular technology but, nevertheless, large scale projects frequently span several technologies.

How can you easily and quickly put together your own project which relies on software components from several different source code repositories ? If your project works out to be cool, how can you easily and quickly share it with other people who may be interested in it ? Now, Linux distribution vendors such as redhat and ubuntu do a great job packaging together a large set of Linux based components – in ubuntu’s case, over 16,000 according to their web site. Great. But what if I myself want to package together some components which I found useful, in a configuration which those Linux distro-vendors don’t currently offer ?

The problem is generally exacerbated if the components which you want to select come from different repositories and projects, having different repository and dependency technologies – CVS, Maven, Subversion, Ant&Ivy, whatever. The problem is also made more difficult because of version dependencies on your installation environment and other components.

In an earlier blog entry, I wrote how assembling software today is reminiscent of childhood experiences with toy building kits, such as Lego. However, there is some danger that in assembling software components from different sources, you end up with a large pile of individual Lego bricks on the floor! There should be a better way of defining and tracking component assemblies across technologies and repositories.

I was excited to see that last week at eclipsecon, AOL’s Lucas McGregor announced the Eclipse Spaces technology project. I guess the essential concept is simple: just in the same way that MySpace provides online storage for friends to share videos and photos and news and events, there is now an opportunity for developers to informally and easily share online interesting component assemblies with one another. The eclipse project process of course offers a formal, process driven approach to collaboratively developing new software between companies, and I guess Apache has something similarly formal between individuals. But Eclipse Spaces suggests something less rigid, in which components which do not necessarily conform to the precise terms of the eclipse.org license, can be informally assembled as personal and individual projects, especially if they come from other repositories outside of eclipse. And since Xdrive is free, this would be a wonderful added service for AOL to provide to the global software community.

I was also delighted to see that Eclipse Spaces is one application of Buckminister’s publishing and materialisation technology for software component assemblies. Henrik Lindberg, Thomas Hallgren and Filip Hrbek are included as the initial set of committers to Eclipse Spaces, all three from Cloudsmith. I’m also delighted to see Malcolm Sparks on the list too, an old IONAian who joined IONA when Ejbhome was acquired in 1999. And Bjorn Freeman-Benson and Ward Cunningham, both from the eclipse foundation. And finally Dennis O’Flynn, the eclipse Corona project lead from Compuware. Looks like a very solid team.

As a global community, we need a far simpler and more elegant way to quickly build, publish, maintain, acquire, materialise and track interesting aggregations of software components, even if they are cross technology. I’ll watch Eclipse Spaces with interest

Back out again in the Valley for part of this week, to attend EclipseCon 2007 in Santa Clara, the annual conference of the world wide open source community contributing to the Eclipse toolkit. Primarily wearing my Cloudsmith hat, of which more in due course..

I was delighted yesterday when IONA formally announced our acquisition of Century 24 Solutions (C24). We’ve been working with these folks for some time and I’ve heard some great things about them both at Board level and directly from our field organisation, although I personally only met Wayne and John at our annual sales kickoff in Boston back in January. The deal was an all cash and although the amount was not specified right now, it will presumably be pretty much deducible soon from the regulatory filings. C24 consists of 12 great people based in London, with a background mainly in the financial services industry. It is exhilarating to have them join, and I believe that we will have a lot of fun together causing a significant impact on the industry.

Just to summarise for you. The C24 technology has both a design and run time aspect. Their design studio builds data models, either from scratch or derived from metadata from various databases, UML models, XML schema, simple text or even extracted from binary. The model can then be tagged with constraints, to ensure the consistency of data such as verifying one category of events or data occur before another. The expressiveness of their consistency rules is quite extraordinary, and is a key feature.

The output generated from the studio is stand alone Java code (a POJO) which parses a data stream natively, applies the various specified constraints, and provides the results via XPath 2. Exceptions – broken data – are filtered out into a separate stream. The XSLT 2 and XQuery actions can be applied natively, bypassing XML, and the resultant subsystem is extremely performant.

Regular long haul flights are inevitably a time for reflection for me, and on the way over here last weekend (Dublin via Schiphol to SFO), I picked up a copy of Ray Kurzweil’s “The Singularity is Near”. Extraordinarily powerful. A core thesis, which will encourage the humanists amongst us, is that intelligence is exponentially more powerful than the physical sciences. In an understanding of the evolution of the universe, he brings the reader through the first epoch of information in atomic structures, physics and chemistry; then information in DNA, leading to biology; then information in neural patterns, and the evolution of brains; then information in non-biological systems, including today’s software and hardware; and then to the singularity – which is the title of the book – of the fusion of intelligence in biology and technology leading to the “waking-up” of the entire universe. Given exponential rates, he places the singularity – the point at which our intelligence and brains merge with external technology – remarkably near: his best guess is just 40 years time. A wild book: read it on your next long haul.

Powerful interconnection competence is fundamental to intelligence and information processing. IONA’s core technologies hitherto have focussed on the fast – high performance and high throughput – interconnection of collaborating services, distributed across a network of machines. The technology elegantly manages the challenge of secure communication, partial failures of the system, disparate technologies, and dynamic reconfiguration. However, self-aware, self-repairing and self-replicating competencies are also fundamental to intelligence. C24 now brings this capability into our core technologies. Data has to be timely and accurate to be of value.

As software practitioners, I think that many of us are contemplating the direction of the industry and its technologies right now. It is a truly fascinating time in the software industry, with the fusion of the power of Service Oriented Architectures (web services, ESBs, mission critical systems), with the rapid emergence and validity of Web 2.0 social technologies and the emphasis on human productivity (including the power of Ruby, PHP, Ajax, etc.). There are also new business models, in Software as a Service, and in open source. There is the emergence of a truly global community of developers and creators, with the growth of the industry in particular in China and India.

It is a great opportunity to architect new structure, and to make money doing so.

The Irish have always been blessed with the “gift of the gab” – a propensity to talk a lot. We even have some data to confirm it, according to the mobile phone operators here in Ireland. Ireland has consistently had one of the highest “average revenue per user” (ARPU) figures internationally for the mobile operators. Now, I can hear some cynics amongst those of you here in Ireland saying the ARPU figures are in fact driven by higher pricing in Ireland, thus “ripping-off” the Irish consumer and business communities. However the operators assert on the contrary it is all because we talk a lot - and they have told the Irish Government so!

In the mobile operator industry, there is continuous competition to first acquire customers, and then retain them and minimise “churn”. You aim to maximise ARPU whilst remaining sufficiently competitive in the market so as to maintain or even increase market share. Optimisation of tariffs is critical: understanding and measuring the tariff efficiency so as to offer attractive tariff plans to your customers, whilst maximising ARPU for their typical usage patterns.

A very naïve approach would be to offer a simple fixed price per month – regardless of actual usage – to each subscriber. Whilst very simple to understand and explain to consumers, I personally doubt whether any operator would survive in the market by offering such a tariff plan. There are a large number of tariff parameters which can be mapped against usage patterns: set-up charges, roaming tariffs, premium rate charges, rollover periods, billing band thresholds, average call rates (including bundling and discounts), flexibility in varying the size and scope of bundled services, rates for calls to other wireless and landline networks, SMS (text) charges, and MMS (multimedia) charges. Operators analyse all of these parameters, in conjunction with the observed traffic patterns generated by customers, and then adjust tariff rates to retain them and acquire new ones.

Many of us in the software sector are closely examining the “Software As A Service” (SaaS) business model, in which software is billed in proportion to actual use, or on demand, rather than the traditional pre-paid perpetual license. The software concerned is usually remotely hosted on an internet server farm, although as an aside to this discussion, I contend that the business model can also be applied to on-premises in-house installed software as well – provided of course the appropriate protections are built in to prevent fraudulent use. The essence of SaaS to me is paying for software in proportion to its usage, regardless of the actual location of the software installation. For enterprise customers – including publicly quoted companies and government systems – secure and verifiable business processes and practices are essential for regulatory approval (such as Sarbennes-Oxley and section 404 in particular) and the physical location of business critical software can become an issue. Remote hosting has benefits and disadvantages; in-house on-premises likewise.

Anyway. End of the aside and back to the main point. My own view is that SaaS is still at a relatively early stage of adoption across the industry, despite the momentum behind leaders such as salesforce.com. In particular, I believe that many SaaS vendors currently offer a very simple charging model: per-user, per-month (PUPM). The number of staff within a customer who access a SaaS offering is measured – or agreed in advance with the Saas provider – and the customer is billed on a monthly basis in proportion to this number.

Wouldn’t it be great if your mobile (or landline) operator charged your company some fixed monthly fee as a multiple of your number of staff using company phones ? Regardless of how much they used their phones and who they called, perhaps on the other side of the planet ? Maybe some operators do have such business models, please let me know (and just to save your keyboard, I already regularly use both Skype and jajah).

Now, perhaps you might argue with me and tell me that the functionality offered by a software package in general is more complex than the range of functions which a telecommunications operator can offer, and therefore it is not fair to compare their respective approaches to tariffs. You might say that a mobile operator needs to offer more sophisticated tariffs than simple PUPM because the industry is so competitive, and thus intelligent and clever bundling of services (SMS and MMS and call rates and roaming charges etc) is critical. A SaaS provider can by contrast just use PUPM, because a vast range of software functions can be offered, and that is the basis for competing, rather than the tariff rates. If you are inclined to that view, then I suspect someone like Nick Carr would disagree with you since he argues that software is rapidly becoming a commodity – like telephones – and that competition based on functionality alone is insufficient, since most software packages of a given genre do pretty much the same thing anyway. Maybe the 80/20 rule also applies too: 80% of the customers in practice only use the same 20% of the functionality available, and are actually pretty happy with it.

I believe that raw PUPM is not good for many customers, and its also not good for a SaaS provider. It is not good for many customers, because the intensity of usage of a SaaS offering may vary considerably across different grades of staff (line staff, field staff, mid-tier managers, and executives); at different time periods (end of quarter and end of year may drive more activity than at other periods); and in different functions (for example marketing campaign vs. pre-sales and lead qualification vs. negotiation and sales vs. customer-support vs. solution fulfillment). The same considerations also make PUPM bad for a SaaS provider: I strongly suspect many SaaS providers in practice leave money on the table as they negotiate under pressure to discount on their standard PUPM rate under such circumstances.

In the telecommunications industry, billing and rating engines are used to capture and describe an operator’s tariff policies, as a series of rules applied to traffic activity records in the network. There would seem an opportunity to apply the same general approach to the SaaS business model: can a SaaS provider use billing and rating technology applied to software activity records (business transactions, business events, session activity, etc) to implement a more sophisticated tariff policy than naïve PUPM ?

In fact, that is precisely what LeCayla does, and that is why I was delighted when I was asked to take on the non executive Chairman role last November. In my view SaaS providers are increasingly unlikely to be able to compete on functionality alone – such software is becoming a commodity, in Nick Carr’s terms.

As Conor Halpin, the LeCayla founder and CEO says: “If you want to maximise the value from each customer then you need a pricing model that captures marginal value.”

The ability to compete on tariff structures - by understanding what customers usage patterns actually are, and devising and applying tariff plans based on business events and activity - is likely to be the basis for a successful SaaS provider, thus increasing value to customers, and increasing market share.

After all, mobile operators have honed their skills to a fine art. They understand what customers usage patterns actually are, and have devised and applied tariff plans based on customers events and activity, thus increasing value to customers, and increasing market share. They understand how much we talk here in Ireland.