chris horn's blog

Like some other teenagers of my generation in the seventies, one of my hobbies at the time was faithfully constructing scale models from plastic kits from Airfix, Revell, Historex etc. As I developed my skills and interest, I took great pride in adding extra levels of detail – particularly for ship kits. Eventually I converted particular kits so as to model an aircraft, ship or vehicle not directly available as a standard kit, by moulding balsa, crafting acetate sheet and so on – for example building a twin engine, triple tail fin Avro Manchester from the Airfix kit of the successor four engine Lancaster. The monthly Airfix Magazine was a great source of designs and examples, and I still have a shelf full of back copies here in my office at home.

When I was an engineering student, I took a similar interest in building my own, albeit fairly simple, analogue electronic circuits – the usual things such as oscillators, radios, fire/smoke detectors, etc. Of course, as with converting plastic kits, having a good stock of spare parts - scavenged from broken electronic circuits – was useful. I used to keep them sorted into empty margarine tubs on my shelves.

In more recent years, I became more and more involved in computers, from the digital gates and circuits up to microcode, assembler, compiler/interpreter systems and finally full systems and applications. I was fascinated when they became affordable as home systems, and played around adding extension cards and interfaces to my first PC. I was initially really impressed by Dell when I saw their range of alternative pre-configured systems for direct order across the internet. Dell used to have a plant in Bray just a few miles away from my home, and I have bought many systems over the years from them for home, school and charitable use.

But I’ve often thought there must be quite a few people out there like me who aren’t quite happy with a specific system they buy from Dell (or indeed any other supplier). Although I carefully chose a configuration before buying, still afterwards I frequently tailor my purchase further with additional cards, expansions and options. I also wonder whether somebody out there has already built the perfect system for creative digital photography, or multi-screen home media management, or whatever. Or whether anyone would be interested if I could publish my own designs and configurations. Or whether any configurations would be commercially interesting, for example for specialist systems such as creative digital media design.

Reflecting on my fads above, the common aspects are a desire to build interesting variations of basic designs; from additional parts either built myself, or stock-piled somewhere (not necessarily in margarine tubs!), or purchased elsewhere; to look at designs published by others, and perhaps even to publish my own.

Well, that’s what Cloudsmith is doing for the world of software. One way to think of Cloudsmith is as a derivation of Dell’s world, for software. You can browse an online catalogue at Cloudsmith and see what “distros” – configurations - of software components are available. If you want to chose any particular one, you click it and it “materializes” – ie downloads its parts, and then automatically assembles them together – onto your machine. And like getting a system from Dell, the different components of your configuration will usually come from different “repos” (repositories - think Dell sub-suppliers) around the world. So rather than just a single download of a binary file, a materialization will usually automatically fetch multiple files, from multiple places, and assemble them automatically for you on your machine as a complete system. The distro is thus “virtual”: it is not monolithic (like an old download) and its parts are not hosted at Cloudsmith (any internet repo can contribute).

If you like what you find, you can send others the “Cloudlink” you used. When they click it, the same components will materialize from the cloud of components available in multiple repos across the internet, and assemble and install on their machine too. So, you can materialize something specific without having to yourself connect to the Cloudsmith site and without having to search the online catalogue for it – just get the Cloudlink for it from someone else via an email or blog or whatever, and the magic will happen when you click.

Whats kinda nice about the Redhats of this world is that they pre-assemble a large collection of software components for you as a full Linux system. Whats nice about Cloudsmith is that if you build your own interesting set of software components, then you can publish that configuration at Cloudsmith, and reliably make it available for other people to use. It need not of course be as complex as a full operating system: just a nice application or tool or subsystem for a particular use which you have put together, using your own design or as an interesting derivation from somebody else's published distro. You can publish novel and interesting things that you’ve done, for others to take a look at and possibly use – no longer are you or they tied just to the pre-assembled varieties which the software equivalents of Dell provide. And if you have components from a new repo which Cloudsmith does not yet know about, you yourself can simply add that repo to Cloudsmith’s map of the world – it is the Cloudsmith software equivalent to introducing Dell to a new sub-supplier.

Why hasn’t a service like Cloudsmith existed up to now ? One reason is that although there has been a proliferation of useful re-useable software components developed and published around the world in many online repos, there has equally been a proliferation of software version control, make and build systems. It is difficult to justify re-engineering everything globally to use the same version control and build technology. IMHO it is impossible to impose one build or version control technology, or one IDE, on everyone: and it is short-sighted to expect everyone else to use the one that you happen to think is best. However, with some thought and help, it is possible for systems like Cloudsmith to automatically, on the fly, interpret the meta-information in all these version control and build system technologies, and thus automatically derive a global perspective and capability across many different repos.

But how do you know that somebody else’s published configuration is any good, and trustworthy? Well how does a plastic modeler or an electronics hobbyist know that somebody else’s design is any good ? Part of the answer is the popularity of the design – how many others are using it, and what do they say about it – and part of it is the reputation of the designer. Social community sites on the internet in general work because most people are ethical, and many people can watch, observe, comment and mend where necessary information which is incorrect.

Assembling and publishing new configurations of components is fun. Is it however just for hobbyists and software hot-rodding ? I believe that many commercial organizations will find interest in discovering and contributing to software configurations and assemblies that add real value in their business domains. I think enterprise managers will find reassurance not only in the popularity of certain distros, but also because they can be reassured that precisely the same distro (and bill of materials) is reliably installed on every machine under their control, and because component updates can be notified and controlled. I think vendor product managers and technology strategists will find interest in the concept of being able to build their own private “cloudspaces” for their licensed customer communities, to manage and distribute software in a controllable way.

Now, let me put my hand up and confess that all of this isn’t yet as smooth as we would like at Cloudsmith just right at the moment: it is an alpha version for the community to experiment with and give us feedback on. The materialization wizard, the wizard to build a cloudlink are IMHO both pretty good and straightforward to use. Private and public cloudspaces are ready to use. But today, the publishing wizard and the wizard to register a new repo are IMHO a bit clunky: we’re improving them right now and we expect better versions within a couple of weeks. Our online documentation is being improved. However, if you’re interested, contact us and we’ll talk you through online and get you involved in the Cloudsmith community.

In the 70s I was a postgraduate student at TCD Computer Science, and interested in digital satellite networking and local area networking technologies. I spent some time studying the design of the Cambridge Ring and other token ring networks. I was then completely inspired by the emergence of CSMSA/CD Ethernet algorithm (in turn derived from the earlier ALOHAnet radio broadcast network) whereby instead of carefully synchronized access to the shared medium, senders are optimistically allowed to transmit whenever they wish: if interference occurs on the medium because of simultaneous transmission, then a randomized backoff procedure allows the system to recover.

I was reminded of all this when I was chatting last week to Lawrence Cowsar, head of Bell Labs Ireland and CTO of the CTVR initiative (I'm chair at CTVR). Sometimes interesting systems result from relaxing constraints and encouraging established taboos to be challenged: for Ethernet, the taboo of careful synchronized access was replaced by unconstrained transmission initiation.

Lawrence, over dinner, was bemoaning the fact that academic research funding is usually and inevitably given for highly focused, highly specialist, uni-disciplinary research. As a result, although the majority of postgraduate researchers, PhDs and post-doctorates may have some experience in team work, that team work is only within the constraints of their own particular research area, under the guidance in most cases of an appropriately myopic faculty member.

By contrast, in general in industry, and in Bell Labs as one example, researchers pro-actively participate in inter-disciplinary teams. This in general seems to be in complete contrast to most academic research. One reason this seems to be is that in general it is difficult to find funding agencies willing to invest in real inter-disciplinary projects, for which multiple research laboratories and multiple terminologies and backgrounds are involved – in general proposals for such research are peer reviewed by research specialists who in turn emphasize uni-disciplinary and singular focus as the best way that their own particular state of the art can be advanced.

I was also reminded of the inertia towards “interference” of disciplines and research when chatting to Lawrence about the technology and algorithms of co-channel management for mobile phone (cellular phone, if you are US) networks. In today’s mobile networks, transmitter masts are carefully erected across a city or landscape so as to minimize interference between masts which reuse the same radio frequencies. This co-channel management challenge is usually resolved by map-colouring algorithms, which allocate frequencies to masts during the network design so as to minimize interference. Sometimes, I smiled to myself, do research proposers seeking financial support from various funding agencies use a map colouring algorithm to minimize the overlaps between their work and so optimize their chances of funding ?

CTVR, and the other CSETs funded by SFI, really are inter-disciplinary: multiple laboratories, academic and industrial, with a very large set of skills and backgrounds. Terminology and a common ability to communicate and collaborate were definitely a problem in the early stage of CTVR. However, we are now seeing highly interesting results which IMHO could only have emerged specifically because of the inter-disciplinary nature of CTVR.

One example is collaboration between the thermal (heat) management research team in the University of Limerick, the radio frequency (RF) and dynamic spectrum research team in NUI Maynooth, the software radio team in TCD, and finally the constraint and optimization analysts at UCC (a full list of CTVR partners is here). It turns out that the taboo of pair-wise interference minimization for co-channel algorithms for mobile (cellular) phone networks can be broken. In fact, channels need not be carefully separated, and interference can be allowed. Using insights from “temperature interference” in thermal management systems, instead of pair-wise constraints between transmitters, a single constraint can be applied to all transmitters whose signals can still be reliably discerned by a receiver. Channels can then be allocated in mobile networks in such a way that has the commercial benefit of reducing the numbers of transmission masts and equipment required.

Who would have thought that research into heat management would lead to an interesting new way of allocating channel frequencies in a mobile phone network ?

Another example of multi-disciplinary work is the application of well understood results in the radio frequency and dynamic spectrum field to the relatively younger area of photonics and management of light, for next generation 100Gigabit Ethernet – work being led by Tyndall at UCC, but for which the NUI Maynooth RF engineers have been able to bring remarkable insight.

In my view – and I believe I speak for Lawrence too – some of the ablest and best researchers which industry can hire come from those postgraduates and post-doctorates who have truly experienced and innovated in inter-disciplinary research. Taboos can sometimes be best challenged and overcome by researchers from outside the immediate specialist domain in question. And yet, world-wide, much academic research seems structurally resistant to such ways of working.