chris horn's blog

A core theme of Web 2.0 is the collective wisdom that results from the network effects of sharing across the community. The wisdom of the crowd is sometimes more than the sum of the individuals therein. Wikipedia is one prime example, since collective knowledge can trigger insights for individuals, which can then augment the collective wisdom: the added knowledge which would not arise if the group did not collaborate together. I recently wrote a blog entry about a similar phenomenon in multi-disciplinary research. Group and individual reasoning can positively feed off each other.

One of the dichotomies of the Web 2.0 phenomenon is on one hand, the power of the group and the collective knowledge of the crowd; and yet on the other the significance of each individual. Yes, collective knowledge such as Wikipedia and del.icio.us (and of course Google!) have emerged, but blogs written and commented by individuals are still a fundamental force. Some may argue that indeed the information in specific blogs written by certain individuals is more valuable, accurate and reliable than that in shared wiki repositories maintained by the amorphous net community. There is room for both individuals and the crowd.

In the race to be successful on the web, the focus is on attracting and retaining users, eyeballs and clicks. For the promoters of a web site - whether a commercial venture, or just a worthy cause for society at large - building and growing a “market” is key. Monitoring usage patterns, listening and reacting to user feedback, all builds momentum: hopefully a tipping point is passed, and the network effect of market momentum reinforces the popularity and acceptance. However: does a mass market strategy play only to the crowd, and not to the individuals therein ?

For Web 2.0 , Chris Anderson introduced the term “long tail” to emphasise deploying:

“customer-self service and algorithmic data management to reach out to the entire web, to the edges and not just the center, to the long tail and not just the head”.

As noted by an Amazon employee quoted by Wikipedia: “We sold more books today that didn't sell at all yesterday, than we sold today of all the books that did sell yesterday” -- read that slowly to yourself again if you hadn’t come across it before..

Further, in his discussion on technology market dynamics, Christensen noted:

“Simply put, when the best firms succeeded, they did so because they listened responsively to their customers and invested aggressively in the technology, products, and manufacturing capabilities that satisfied their customers' next-generation needs. But, paradoxically, when the best firms subsequently failed, it was for the same reasons--they listened responsively to their customers and invested aggressively in the technology, products, and manufacturing capabilities that satisfied their customers' next-generation needs ….But the problem established firms seem unable to confront successfully is that of downward vision and mobility, in terms of the trajectory map….”

Success – at least in some industries, such as the disk drive and earth moving machinery industries which Christensen documents – can equally lay the foundation for failure. Group and individual behaviour can play off each other, creating emerging forces from below.

Combining the Anderson’s exhortation to reach out to the edge, with the warning by Christensen not to be outflanked by emerging disruptive technologies from below, it would seem wise to ensure that a Web 2.0 market strategy explicitly recognises the potential of the individual, as well as the mass market of the group. Even more important is to react to dynamics in the market and to changing tastes. The observation is: that in a global market of mass consumerism, the ability to cater for the vast number of changing individual personal tastes and desires – to personalise and dynamically tailor your products and services – may be much more commercially – and socially – valuable, than volume plays of standard products and services to an anonymous amorphous group.

The long tail changes: it wags. The universe of micro-markets in the long tail is worth addressing – this is Anderson’s observation. In addition, changes in a micro-market may in due course influence the mass market (and blind side you) – this is in essence Christensen’s observation.

Ajit Jaokar was one of the contributors at the Fidelity event in Galway, and he reminded the audience of Tim O’Reilly’s characterization of the Web 2.0 phenomenon:

• the web as a platform;
• harnessing collective intelligence;
• data as the next Intel inside;
• the end of the software release cycle;
• software above the level of a single device; and
• rich user experiences.

I thought about each of these six facets as I reflected on the dynamics of the micro-markets, and list some (not all!) of my observations below: how does Web 2.0 address the wagging long tail ?

One common way of reaching out to the long tail, and monitoring its movements, is “data as the next Intel inside”. Google exploits what its itself calls the ”uniquely democratic nature of the web” to derive data on the click activity of millions of users to drive its PageRank algorithm. I suspect that social networking sites, such as MySpace, Facebook, Bebo and LinkedIn, could likewise exploit their data about networks of people. I’ve always wondered in passing how data regulators, such as the Irish Data Protection Commissioner, view such click gathering and social networking activities…

As Web 2.0 embraces beyond “the level of a single device” – which was Ajit’s theme of his presentation - I observe that telecommunications operators, such as Vodafone, O2 and T-Mobile, collect and record quantities of data, ironically in a large part due to regulatory requirements of data retention legislation in jurisdictions such as Ireland. These vast data archives capture about social communication patterns of populations – it is not just the social networking sites which have such data! There are opportunities, and privacy risks, to exploit these vast patterns to provide commercial value. One can imagine intelligence being generated by mobile phone operators to enable targeted marketing by third parties and content providers…

Quietly collecting mouse click activity, or analyzing phone call patterns and triangularising the locations of their owners, are data driven ways of using to drive for the long tail. Importantly, the data is dynamic and therefore so also is the derived collective intelligence: as trends emerge, or change, so can the offerings tailored for particular sets of individuals. But it seems to me that these approaches may wear a clandestine cloak: there are more explicit ways of using collective intelligence for the wagging long tail.

A successful web site needs stickiness to retain, and track, its audience, including the wagging long tail. A social networking site such as those above in part generates its stickiness because of the network effect of having ones friends, co-workers and colleagues using the same site: most people think it is too cumbersome to re-register oneself and encourage one’s cohort elsewhere. IMHO, the resulting stickiness is in fact rather superficial, rather than systemic: it consequently threatens any purported fiscal value for the site. The SIOC project at DERI in Galway underlines the benefits, and business consequences, of seamless federation of online communities. Stickiness should not be driven by re-registration inertia, but rather by community value and impact - which is one reason why I believe Ammado will succeed.

Another strategy to capture the long tail wags is “harnessing collective intelligence” – another of O’Reilly’s Web 2.0 facets. One explicit way is to allow individuals to define their own collections of interesting items, and then to share and discuss them with other like-minded souls. A simple way to do this is just by tagging items: for example, photos tagged with “Galway” on Flickr will appeal to certain people. But tagging rather quickly loses its impact when trying to find items with a combination of tags, requiring tedious trials of various search criteria; and tag synonyms are a problem in a global world: try “football” in del.icio.us as an example – or even “Galway Hooker” in Flickr!

Rather than expecting people to work out the correct search and query expression across a set of tag values for what they actually want, another way is to let people explicitly define and share their own collections. Hobbyists – such as stamp collectors – have been doing this for years, and a craft worker likewise learns what the set of right tools are for the job from the experiences of his trade. Enabling individuals – and micro-markets and even the crowd - to share what works well together when, is a positive tactic for managing the wagging long tail, and which I wrote about recently in the context of software configurations and Cloudsmith.

When doing a little research for this blog entry, I came across Dan Bricklin’s interesting blog entry on When the Long Tail Wags The Dog. His theme is that “must have” items have more value than those which are less likely to fit the job at hand; and that general purpose items are likely to have most value since they can entertain the dog as well as its long tail. While I don’t disagree, my thoughts about the dynamics of the long tail, and how to work with within it, are a slightly different emphasis.

Reinforcing Bricklin, being relevant to the individuals and small groups in the long tail requires customization and tailoring of general purpose offerings. In the enterprise software space, software vendors have traditionally worked with systems integrators to tailor more appropriate solutions to different niches of the market. With componentization in the software industry, in principle useful aggregations of different constituent parts can more likely be used to tailor specific solutions. Collective intelligence can be harnessed by explicitly sharing these aggregations, as I indicated above.

However, once delivered, installed and put into use, assemblies of software components in production environments have in the past usually been relatively static. The challenge is that the long tail wags: micro-markets and even mass markets change, and production systems need to more easily do so too. One promise of the dynamic module environment of OSGi is to enable dynamic evolution: if collective intelligence can be dynamic as trends emerge, or change, and then suggest new offerings tailored for particular sets of individuals, then perhaps also can production software assemblies in business and enterprise environments be dynamically adjusted. This is a theme which we are working upon in IONA, particularly in the context of our highly dynamic plug-in architecture for Artix and our open source FUSE offerings – see Eric Newcomer’s blog.

But if the world is dynamic, and contains a universe of micro-markets as well as a mass, can offerings not only be tailored so as to be a good fit, but also be priced attractively for each ? What when micro-markets change ? As I wrote about, one industry which faces these challenges is the mobile/cell phone operators, for whom intelligent and responsive bundling of services (SMS and MMS and call rates and roaming charges etc) is competitively critical. The ability to generalise this approach into the world of software components and packages is of interest and relevance to the wagging long tail. LeCayla is one company building a common approach to the issue of dynamically rolling out new charge and billing rates so that software vendors can competitively foster specific micro-markets.

Let me summarise: the internet as a platform enables a global market to be addressed. Web 2.0 uses collective intelligence to play in both the mass markets, and the long tail of numerous micro-markets. There is room for both individuals and the crowd, and group and individual reasoning can positively feed off each other. – creating the ever lurking possibility of being disrupted from below. Scalably addressing both the mass market and the long tail is however not the complete issue: markets change, the long tail wags, and scalably addressing dynamic markets is even harder – but possible.

Footnote: my 7 month old Alsatian, Charlie, has yet to grow into his long long tail. He wags it a lot.

Cloudsmith

An upgrade to the web site at www.cloudsmith.com – more features, particularly so that you can easily map your own repository, better documentation. The site is generating considerable interest in both the open source community, and some commercial firms seeking interesting new ways to distribute and support their software.

Note that the next major release of the Eclipse development environment – the Ganymede release - will be built using Buckminster: the team behind Buckminster also have built Cloudsmith, as I discussed in some earlier posts..

Science Gallery

Opening in Feburary next, with the LIGHTWAVE festival and showcase, and getting some good coverage eg in Nature and the Irish Times recently. Watch the blog at www.sciencegallery.org for further news and announcements.

UNICEF

A great new video at http://www.youtube.com/user/UNICEFIreland - featuring Irish children with Irish celebrities Andrea Corr, Colin Farrell, Marian Finucane, Stephen Rea and Ryan Tubridy. Our UNICEF colleagues in the US Fund produced a similar (US centric) video last year, and the Irish National Committee have borrowed their idea, with permission.

“Every single child deserves to live”.

Like some other teenagers of my generation in the seventies, one of my hobbies at the time was faithfully constructing scale models from plastic kits from Airfix, Revell, Historex etc. As I developed my skills and interest, I took great pride in adding extra levels of detail – particularly for ship kits. Eventually I converted particular kits so as to model an aircraft, ship or vehicle not directly available as a standard kit, by moulding balsa, crafting acetate sheet and so on – for example building a twin engine, triple tail fin Avro Manchester from the Airfix kit of the successor four engine Lancaster. The monthly Airfix Magazine was a great source of designs and examples, and I still have a shelf full of back copies here in my office at home.

When I was an engineering student, I took a similar interest in building my own, albeit fairly simple, analogue electronic circuits – the usual things such as oscillators, radios, fire/smoke detectors, etc. Of course, as with converting plastic kits, having a good stock of spare parts - scavenged from broken electronic circuits – was useful. I used to keep them sorted into empty margarine tubs on my shelves.

In more recent years, I became more and more involved in computers, from the digital gates and circuits up to microcode, assembler, compiler/interpreter systems and finally full systems and applications. I was fascinated when they became affordable as home systems, and played around adding extension cards and interfaces to my first PC. I was initially really impressed by Dell when I saw their range of alternative pre-configured systems for direct order across the internet. Dell used to have a plant in Bray just a few miles away from my home, and I have bought many systems over the years from them for home, school and charitable use.

But I’ve often thought there must be quite a few people out there like me who aren’t quite happy with a specific system they buy from Dell (or indeed any other supplier). Although I carefully chose a configuration before buying, still afterwards I frequently tailor my purchase further with additional cards, expansions and options. I also wonder whether somebody out there has already built the perfect system for creative digital photography, or multi-screen home media management, or whatever. Or whether anyone would be interested if I could publish my own designs and configurations. Or whether any configurations would be commercially interesting, for example for specialist systems such as creative digital media design.

Reflecting on my fads above, the common aspects are a desire to build interesting variations of basic designs; from additional parts either built myself, or stock-piled somewhere (not necessarily in margarine tubs!), or purchased elsewhere; to look at designs published by others, and perhaps even to publish my own.

Well, that’s what Cloudsmith is doing for the world of software. One way to think of Cloudsmith is as a derivation of Dell’s world, for software. You can browse an online catalogue at Cloudsmith and see what “distros” – configurations - of software components are available. If you want to chose any particular one, you click it and it “materializes” – ie downloads its parts, and then automatically assembles them together – onto your machine. And like getting a system from Dell, the different components of your configuration will usually come from different “repos” (repositories - think Dell sub-suppliers) around the world. So rather than just a single download of a binary file, a materialization will usually automatically fetch multiple files, from multiple places, and assemble them automatically for you on your machine as a complete system. The distro is thus “virtual”: it is not monolithic (like an old download) and its parts are not hosted at Cloudsmith (any internet repo can contribute).

If you like what you find, you can send others the “Cloudlink” you used. When they click it, the same components will materialize from the cloud of components available in multiple repos across the internet, and assemble and install on their machine too. So, you can materialize something specific without having to yourself connect to the Cloudsmith site and without having to search the online catalogue for it – just get the Cloudlink for it from someone else via an email or blog or whatever, and the magic will happen when you click.

Whats kinda nice about the Redhats of this world is that they pre-assemble a large collection of software components for you as a full Linux system. Whats nice about Cloudsmith is that if you build your own interesting set of software components, then you can publish that configuration at Cloudsmith, and reliably make it available for other people to use. It need not of course be as complex as a full operating system: just a nice application or tool or subsystem for a particular use which you have put together, using your own design or as an interesting derivation from somebody else's published distro. You can publish novel and interesting things that you’ve done, for others to take a look at and possibly use – no longer are you or they tied just to the pre-assembled varieties which the software equivalents of Dell provide. And if you have components from a new repo which Cloudsmith does not yet know about, you yourself can simply add that repo to Cloudsmith’s map of the world – it is the Cloudsmith software equivalent to introducing Dell to a new sub-supplier.

Why hasn’t a service like Cloudsmith existed up to now ? One reason is that although there has been a proliferation of useful re-useable software components developed and published around the world in many online repos, there has equally been a proliferation of software version control, make and build systems. It is difficult to justify re-engineering everything globally to use the same version control and build technology. IMHO it is impossible to impose one build or version control technology, or one IDE, on everyone: and it is short-sighted to expect everyone else to use the one that you happen to think is best. However, with some thought and help, it is possible for systems like Cloudsmith to automatically, on the fly, interpret the meta-information in all these version control and build system technologies, and thus automatically derive a global perspective and capability across many different repos.

But how do you know that somebody else’s published configuration is any good, and trustworthy? Well how does a plastic modeler or an electronics hobbyist know that somebody else’s design is any good ? Part of the answer is the popularity of the design – how many others are using it, and what do they say about it – and part of it is the reputation of the designer. Social community sites on the internet in general work because most people are ethical, and many people can watch, observe, comment and mend where necessary information which is incorrect.

Assembling and publishing new configurations of components is fun. Is it however just for hobbyists and software hot-rodding ? I believe that many commercial organizations will find interest in discovering and contributing to software configurations and assemblies that add real value in their business domains. I think enterprise managers will find reassurance not only in the popularity of certain distros, but also because they can be reassured that precisely the same distro (and bill of materials) is reliably installed on every machine under their control, and because component updates can be notified and controlled. I think vendor product managers and technology strategists will find interest in the concept of being able to build their own private “cloudspaces” for their licensed customer communities, to manage and distribute software in a controllable way.

Now, let me put my hand up and confess that all of this isn’t yet as smooth as we would like at Cloudsmith just right at the moment: it is an alpha version for the community to experiment with and give us feedback on. The materialization wizard, the wizard to build a cloudlink are IMHO both pretty good and straightforward to use. Private and public cloudspaces are ready to use. But today, the publishing wizard and the wizard to register a new repo are IMHO a bit clunky: we’re improving them right now and we expect better versions within a couple of weeks. Our online documentation is being improved. However, if you’re interested, contact us and we’ll talk you through online and get you involved in the Cloudsmith community.

As some of you may know, I regularly holiday just outside Roundstone in Connemara. I’ve just come back to Dublin yesterday after some time there again.

If you haven’t yet been to the west of Ireland, I think one of the most striking things is the web of small stone walls that embrace the fields, pastures, meadows, boglands and tracts. Each is made by hand, and almost always as dry stone walls without mortar. They usually are as a result of clearing granite stones and rubble from the fields, and are economic: not requiring mortar, they do not suffer from frost attack, and so little maintenance is needed. At first sight, they all appear similar, but in fact there are different construction styles, with single, double and combination walls as the basic classification. They are malleable: walls can be easily moved and re-configured – gates are not strictly necessary since a few stones can easily be removed and put back again to, for example, let cattle through. Patrick McAfee’s book and website are a very readable commentary.

From ground level, the profusion of little stone walls can appear as a complex pattern, perhaps even fractal-like. But viewed from a high point, perspective reveals the logic of the landscape and the paths – the boreens – lined with walls either side, gently meandering to distant places.

I came back to Dublin last night, and this morning read Martin Banks’, of Reg Developer, excellent overview of Buckminster. Since I first wrote about Buckminster, the team has considerably improved the tool, including the documentation kit. Martin’s article makes use of a bricklaying analogy, and I guess I hinted in my own blog entry that if you are to build structures from re-usable bricks, then it might be useful to have a web site somewhere at which various designs could be published found and compared…

Well, also while I was away in Connemara, www.cloudsmith.com went live. Many of those on the Buckminster team have collaborated to put the site together, and the initial incarnation of the site certainly turned out to have richer functionality than I myself expected in a first iteration. There is a fairly detailed overview on the About Cloudsmith page, but in summary:

• Cloudsmith keeps meta-data about assemblies of software components.

• Software components can be sourced from any number of public and private repositories worldwide. Of course, components from private repositories are only available to those duly authorized to use them.

• Cloudsmith does not store the components themselves: but it knows where they are worldwide and how it can access them in the appropriate repository formats.

• A software publisher – an individual, project, or company – can register one or more specific software component assemblies with Cloudsmith.

• A software consumer – an individual, project, or company – can search and browse for available assemblies; and can readily download and install any particular one – “materialize” in Cloudsmith-speak – onto his local machine (or indeed another machine if appropriately authorized).

In effect, Cloudsmith is building a global map of software components (in various forms: source, binary, versioned, and optionally with test suites, documentation and license agreements). Professional software developers - individually or in a community project or working on a commercial offering – can publish interesting new assemblies of components, sourced across one or more repositories.

One of the neatest capabilities of Cloudsmith is a Cloudlink. A Cloudlink is simply a URL: it can be sent in an email, or given in a blog or whatever. When a Cloudlink is clicked, the software assembly which it denotes is then materialized without further intervention, onto the local machine. This gives a very simple download mechanism: publish a Cloudlink, and anyone clicking on it within a recent-vintage web browser can download your software. In practice, when a Cloudlink is clicked, behind the scenes the Cloudsmith site is contacted, and it resolves the differences between the assembly of software components identified by the Cloudlink, and those already available on the local machine, and then fetches (as appropriate from various repositories worldwide) and downloads the missing components.

Cloudlinking in turn enables “virtual distributions”. A software publisher can create a virtual distro, whose components reside across multiple (eg open source) projects and repositories: materializing a virtual distro requires nothing more than a web browser.

If your project is looking for a simple way to make its software available to the worldwide community; if your project is itself using software from multiple sources and multiple projects; if you want to keep your community regularly updated with patches and extensions; if you want to manage installation and distribution processes; then Cloudsmith should be worth taking a look.

Software components, and configurations and assemblies of them, are very malleable. It is relatively easy to define new interesting configurations, as well as new components. Looking at the world wide activity, and the multitude of repositories and projects, it is easy to become overwhelmed. It is possible to detect patterns, and different styles of construction, but sometimes it can be very confusing to see overall themes, to understand how other people are using configurations, and what changes have occurred.

I’m reminded of Connemara’s dry stone walls. They are numerous, wonderful, simple, easily changed, easily re-built, easily maintained, and as a result have lasted for decades. But in the landscape and up close, they are confusing to absorb and see the overall picture. The perspective of height gives clarity.

Cloudsmith is giving clarity to the construction of assemblies of software components.

It is an interesting time for the world of “Software As A Service”.

At some sort of cerebral level, Dell has been an inspiration for those contemplating a SaaS go-to-market strategy. Dell was renowned in the PC industry for largely avoiding the cost of enterprise sales and retail distribution channels, by instead connecting directly to end purchasers – whether domestic, small/medium business, or enterprise – as much as possible. It is very interesting thus that last week Dell announced, for the first time in 15 years, a retail deal: this time, with Wal-Mart to put some of its low end products directly onto the retailer’s shelves, as its “first step” into a retail channel. Dell apparently needs a distribution platform – and a retailer which 90% of American shoppers use is more attractive than most other retailers – through which to advertise some of its products.

Also this week, Ray Ozzie of Microsoft has stated that it is no longer about “Software As A Service” but “Software And A Service”! He was speaking at the Mix07 developers conference, and primarily promoting the Silverlight technology for Rich Internet Applications. Silverlight will be a runtime for .NET in various browsers – IE, Firefox and Safari – and compete with Flash, AJAX and Javascript to – in Microsoft’s view – provide a much richer video and interactive graphics experience for end users. Ozzie noted that (“fat”) client software not only supports offline usage, but also offers “privacy, empowerment, anonymity and freedom” compared to the “monitoring, auditing and creepy behaviour” of some online services. I guess this was a swipe in particular at Eric Schmidt at Google who is currently promoting substantially enhanced search experiences for end-users, albeit with the consequence of maintaining (private) information about each user.

An interesting part of Ozzie’s presentation was his announcement that Microsoft Live is offering a Silverlight Streaming service providing 4Gbytes of free storage per Silverlight developer, so as to encourage the development and hosting of Silverlight applications. So, if you are a S(as/and)aS developer or independent software vendor (ISV), Microsoft is potentially offering you a rapid way to scale up to address a massive online audience, based on MSN and now not just IE but other browsers as well: up to 1 million unique end users can access your application for free (above that, its 25c per unique user). It will be interesting to see how Google responds..

IMHO we will continue to see a mix of pure software-as-a-service applications (with thin clients); online services with companion client applications (including rich internet access), and fat clients accessing remote services from time to time. There will continue to be a spectrum of configurations.

Regardless, I believe that a critical aspect of S(as/and)aS applications is the ability to scale self-service. Whether you use Silverlight Streaming, Google, Salesforce.com’s Apex or even Wal-Mart (as Dell is doing) to reach a potential audience of millions, I believe that a S(as/and)aS application will be unsuccessful if scaling of its adoption requires manual intervention for every transaction.

Self-service is key to S(as/and)aS – whilst at the same time ensuring that the customer’s experience is happy and helpful. Dell’s appearance in Wal-Mart may in part be due to reports of poor customer service with a purely online self-service store.

One example of online software self-service is of course the open source community. What could be more self-service than downloading source code, and playing and building it yourself ? A common open source business model is to extend self-service with technical support and training. But in turn, this customer support will best be implemented by at least some degree of self-service: witness IONA’s various support offerings for its Celtix family of open source products, including a self-service knowledge base alongside telephone and email support.

LeCayla’s self-service philosophy goes further. In providing SaaS metering and billing, LeCayla’s technology has to address two audiences. The first is end-users of SaaS offerings: LeCayla measures usage, and generates bills and invoices according to actual usage, and in accordance with specific business rules defined by a particular SaaS ISV or application software provider. Naturally, it is desirable that each end-user may use a self-service interface: eg to check her usage, or to change billing information such as a credit card number.

The second audience for LeCayla is the ISVs who want to use LeCayla to meter and bill for use of their software products. Each such ISV registers business rules (eg pricing information, usage tiers, etc) into LeCayla. Furthermore, each such ISV may wish to subsequently change its own business rules at any time – for example for a pricing promotion of a particular product within a particular geography. Naturally it is desirable that not only should end-users have self service to their own usage and billing information; but also that each ISV should also likewise have self-service to its own business rules, as well as to its market adoption metrics and usage information.

For Cloudsmith – the third software company in which I am involved – self-service is also key. As I noted in a previous posting, “sometimes creative developers have different configurations that they seek, or want to define, share and publish to the world. Can interesting new virtual distributions be rapidly defined, communicated and materialized ?”. Cloudsmith will enable developers to self-service find and use interesting software configurations, each frequently materialised from different software repositories (and sometimes using widely different repository technologies and build/make systems). Equally, publishing a new configuration, either to the entire world or to a private community of collaborating developers, will be a self-service activity.

Self-service seems to be intrinsic to scaling software services offered over the internet. Self-serviced services must naturally be scaleable: poor customer support and dissatisfaction will otherwise result. In a self-service, service oriented world, multiple business models are IMHO possible: for example, free access with optional paid-for support and consultancy (IONA’s approach with Celtix as per above); metered usage (LeCayla’s approach); or community based (Cloudsmith’s). I believe that scalable self-service underpins any viable service oriented business model.

Not all S(and/as)aS transactions should be self-service. But unless your S(and/as)aS business model facilitates self-service, then you may be scaleably challenged!

Part of the dilemma with the global software community today is that there are so so many open source repositories and projects. I guess amongst the best known community owned initiatives are Apache, eclipse and SourceForge. But there are very many others, as a quick Google will reveal: for example HotScripts, NetLib, RubyForge, and CodeHaus. Most of course are primarily specific to a particular technology but, nevertheless, large scale projects frequently span several technologies.

How can you easily and quickly put together your own project which relies on software components from several different source code repositories ? If your project works out to be cool, how can you easily and quickly share it with other people who may be interested in it ? Now, Linux distribution vendors such as redhat and ubuntu do a great job packaging together a large set of Linux based components – in ubuntu’s case, over 16,000 according to their web site. Great. But what if I myself want to package together some components which I found useful, in a configuration which those Linux distro-vendors don’t currently offer ?

The problem is generally exacerbated if the components which you want to select come from different repositories and projects, having different repository and dependency technologies – CVS, Maven, Subversion, Ant&Ivy, whatever. The problem is also made more difficult because of version dependencies on your installation environment and other components.

In an earlier blog entry, I wrote how assembling software today is reminiscent of childhood experiences with toy building kits, such as Lego. However, there is some danger that in assembling software components from different sources, you end up with a large pile of individual Lego bricks on the floor! There should be a better way of defining and tracking component assemblies across technologies and repositories.

I was excited to see that last week at eclipsecon, AOL’s Lucas McGregor announced the Eclipse Spaces technology project. I guess the essential concept is simple: just in the same way that MySpace provides online storage for friends to share videos and photos and news and events, there is now an opportunity for developers to informally and easily share online interesting component assemblies with one another. The eclipse project process of course offers a formal, process driven approach to collaboratively developing new software between companies, and I guess Apache has something similarly formal between individuals. But Eclipse Spaces suggests something less rigid, in which components which do not necessarily conform to the precise terms of the eclipse.org license, can be informally assembled as personal and individual projects, especially if they come from other repositories outside of eclipse. And since Xdrive is free, this would be a wonderful added service for AOL to provide to the global software community.

I was also delighted to see that Eclipse Spaces is one application of Buckminister’s publishing and materialisation technology for software component assemblies. Henrik Lindberg, Thomas Hallgren and Filip Hrbek are included as the initial set of committers to Eclipse Spaces, all three from Cloudsmith. I’m also delighted to see Malcolm Sparks on the list too, an old IONAian who joined IONA when Ejbhome was acquired in 1999. And Bjorn Freeman-Benson and Ward Cunningham, both from the eclipse foundation. And finally Dennis O’Flynn, the eclipse Corona project lead from Compuware. Looks like a very solid team.

When I was a kid, my sister and I loved sharing our toys and playing together. She had a farm, with farm buildings and fenced fields; a hospital, complete with a helicopter landing pad for emergency cases; and a dolls house, whose front swung open to reveal a hidden world for miniature furniture and figures. I myself had a garage, which had an inspection bay and an elevator for the cars; docks, with piers and cranes and ships; and an airport, with runways and a passenger terminal, and tiny fuel trucks and baggage carts. We would sprawl ourselves across the play room and build our town, playing as gods as we moved people and animals and freight in our virtual world, from the farm to dock to the airport to the hospital. Of course, we had to try and ignore the differences in scale of our toys – her dolls house dwarfed everything – as well as the bumps and crevasses in our roads, where the edges of the bases of the toys butted up against each other.

In those “black and white days”, as my now 10 year old daughter calls them – the days before colour TV and lcd flat panels – there wasn’t a lot of plastic, and wonderful kits like Lego™ had only just begun appearing. Now my own kids can build towns from Lego kits, with consistent scale across all the buildings, unless of course they choose to make one building much bigger than the others. The roads all smoothly meet up at the junctions, and Lego people and animals and freight all appear wonderfully realistic as scenarios are enthusiastically played out for Dad’s benefit.

What is even more wonderful about toys such as Lego is that children’s fantasies can be cultivated, as they themselves design and construct their own buildings and vehicles and systems, using their imagination to mimic and extend their worlds. They are not limited to the pre-designed model described by each Lego instruction sheet or booklet. Indeed, my teenage son shares designs with his friends, sketching out their own instructions, and photo-copying them for each other. I googled to see whether there is a web site somewhere where people can share their designs and instructions with each other – I could find a few (such as http://staff.science.uva.nl/~leo/lego/ and http://www.telepresence.strath.ac.uk/jen/lego/ and Lego’s own http://factory.lego.com) but no single aggregation: now, there’s an idea if someone hasn’t already done it.

In 1992, when I started building Orbix v1.0 - our CORBA implementation - with John Moreau and Bridget Walsh, the software community was talking about the novel concept of object oriented programming, and some proponents certainly drew parallels with construction kits such as Lego. If civil engineers build using pre-fabricated components, and kids build complex buildings and systems from re-usable Lego bricks, then why can’t the software industry build using a portfolio of re-usable objects and components ? A decade plus later, I guess today we pretty much have learnt this lesson, and much of our complex software artefacts are constructed using re-usable components and common design patterns.

In the “black and white” days, toys were complete. A dolls house was a dolls house. A toy garage was a toy garage. You couldn’t re-assemble it to build something closer to your imaginary perfect garage. A decade ago, software applications and indeed operating systems were likewise monolithic. Sure, you could configure products like SAP R/3 and SVR4, but you couldn’t delve inside them, learn from them, and build systems which were closer to the perfection you craved for your organisation. Today, open source, and software componentisation, has changed the industry, and so you can now assemble and build tailored – and even generally useful – software yourself.

However, now the interesting thing, I think anyway, is where is our – I mean the software world’s – equivalent of those Lego instruction sheets and booklets, those photo-copied personally designed instructions for new models ? Software design patterns are not really the same thing: a software design pattern gives a “how-to” elegantly solve a commonly occurring scenario, regardless of what you are actually building; a Lego instruction sheet, whether direct from Lego or invented by one of your friends, tells you how to build a complete and finished model.

That’s why I got very enthusiastic when Mitch Sonies and Henrik Lindberg approached me last year to talk about Buckminster and Cloudsmith. Buckminster is open source technology which they have donated to the Eclipse foundation -- although Buckminster is not dependent on Eclipse, a “headless” version also exists. A specific software configuration is captured in an RMAP and CQUERY. I like to think of a CQUERY as akin to a Lego instruction sheet, with the RMAP saying which Lego blocks are in which containers and bags. An CQUERY captures, both explicitly but also implicitly by transitive closure, all the necessary software components – other completed models and even individual bricks – needed to build a fully complete system, or component. Given a CQUERY and RMAP, Buckminster then can actually go ahead and build – or “materialize” in Buckminster parlance – the artefact so described, for you on your machine.

Now, you might say, gee Chris, ‘make’ systems have been doing that even since before you were a developer back in the 90s. Well, Buckminster is a bit more than that. What if you want to build something that depends on components from different software repositories: some repositories inhouse in your company or on your own private computer, and some remote across the internet. What if these repositories use different formats – CVS, maven, clearcase ? What if there are version dependencies between the components ? What if there are alternative repositories for the same component ?

Some of the Eclipse projects are looking very hard at Buckminster, and I understand that some have already adopted it. I think there will be quite a bit of discussion about Buckminster at the forthcoming eclipsecon conference. But I don’t want you to think of Buckminster as an Eclipse-based Java-centric tool. Because it isn’t. Back to the Lego analogy – as well as Lego, not only is there Duplo from the same company, but also construction kits from Young Architects and Brio and Imagibrick and others. In our software industry, we’re similarly plagued by different core technologies, leading to bumps and cravesses as you try to butt together one completed system with another. Buckminster at least solves the problem of defining and materializing useful configurations of software – regardless of the technologies – Java, PHP, C# or whatever - used.

One of the most frustrating things about the software industry today, I think, is that although it is much easier now to build interesting new artefacts and systems from re-usable components, it is quite difficult to communicate these configurations and fit them together with other peoples’. Linux distribution vendors, such as RedHat and Ubuntu, do a fine job selecting a specific configuration of several thousand software components, and warranting to a greater or lesser degree that they should all work together. But, sometimes creative developers have different configurations that they seek, or want to define, share and publish to the world. Can interesting new virtual distributions be rapidly defined, communicated and materialized ?