VenChar

Open Source and the Enterprise Infrastructure Stack

One question I keep thinking about is how much of the enterprise infrastructure stack will open source penetrate?  People like Marc Fleury at JBoss   believe that open source will keep climbing up the stack and move up from application servers to the middleware platform. 

One recent article that does a terrific job of capturing the lifecycle of open source development is Craig James's The Care and Feeding of FOSS.  He believes that open source projects start really taking off when a product reaches the maturity stage:

 

With the slow pace of innovation of the Maturity phase, the FOSS community begins to slowly but     inexorably erode the technical lead held by the commercial offerings. FOSS versions of the technology     may have been present all along, but the pace of innovation during the Expansion phase often left     them in the dust. But now, with the technology mature and the pace of innovation slow, FOSS becomes     the proverbial turtle, plugging along toward the finish line, slow but unstoppable. Feature by feature,     the FOSS developers eat away at the commercial products.

<>

The commercial suppliers are doubly cornered. First, the product is no longer cutting edge, so staffing is reduced and management interest is low.   Since there's little innovation, R&D costs are low, which means profits are high. Developers who want   to innovate are discouraged, because there's little potential return on investment.

Second, the technology has expanded to the logical boundaries, and additional features are less and   less relevant to the core technology. These two factors slow innovation dramatically in the commercial   sector.

Sooner or later, the FOSS product not only matches the commercial products feature-for-feature, but   the nature of open-source software makes the FOSS product more reliable, higher performance and (where   security is a concern) more trusted.

This worldview postulates that the commercial software companies will be the ones who will continue to innovate while the open source community will dominate the more mature technology categories such as operating systems, databases, app servers, etc.

Larry Ellison in an interview that was blogged on Always On has a different take on open source:

The interesting thing is that for an open source product like MySQL to get a lot of traction, they're going to have to walk down the same road that Linux did, which is to get a lot of very large companies to support them. There is this myth that Linux was created and popularized by a bunch of guys who worked by day at hobby shops. Then supposedly they'd go home and program in Linux in their free time. But in fact, the biggest supporters of Linux are businesses like IBM. IBM is not a hobby shop. Oracle, we're not a hobby shop. Hewlett Packard. There are huge companies supporting Linux and the open source movement. MySQL doesn't have that same kind of support behind it. SAP is the first large company to begin to support MySQL, but again, if you compare that landscape to the number of companies that are willing to launch Linux...there are just a bunch more companies supporting Linux. So I don't think you can just paint with a broad brush and say it's "open source versus not open source." It's "open source that has support by the technology industry."

If you believe Ellison's worldview then you would feel that open source can only start climing up the stack if there are large technology players behind it.  I don't buy that.  I am closer in my thinking to Craig James than I am to Larry. 

One concern neither of them mention is the legal indemnification issue.  Authorship in open source by its very nature is muddled and its often hard to know whether contributers provided original code or have polluted it with copyrighted code.  Until the legal indemnification issue is solved or atleast until large companies can stand behind the open source code, most large enterprises are going to feel uncomfortable taking the plunge.

MySQL in Wired

Wired article on MySQL gaining traction - 4000 paying customers (including Yahoo, Google, Caterpillar, UPS), $10MM in sales last year, 40% penetration among database developers. Good example of a low-end disruption. Interesting quote from a Microsoft product manager for the SQL server:

Tom Rizzo, Microsoft's director of product management for SQL Server, said he saw no immediate threat from open source, at least in the database arena.

"Typically, MySQL and other open-source database companies are used in small departments," Rizzo wrote in an e-mail. Developers tend to deploy open source to save the money or hassle of going through a cumbersome IT procurement process.

Rizzo cited the key advantage of the SQL Server over MySQL or other open-source databases as the "ability to provide innovative features that address complex needs of a user."

That's how low-end disruptions work - the product starts out being less full-featured than the existing higher-end products, so that it initially only takes away market share at the low end. As a result the market leader dismisses it as a threat. Over time it proceeds to get better, more full-featured and by the time it has ramped up the technology improvement curve it's already too late to stop the juggernaut.

As an aside, MySQL pursues an interesting dual-license strategy: you can either get the code for free (in which case you have to contribute back any changes you make) or you can pay for a license (in which case you can keep your modifications proprietary).

Localization of software and emerging markets

Interesting article over at CNet (via /.arguing that localization (providing local language support) could turn out to be one of the drivers for adoption of open source software in emerging markets.

For now, such projects are largely curiosities. But analysts say they could present a significant long-term threat to Microsoft's dominance on PC desktops. Regions and language groups that don't have enough of a PC market now to justify development of proprietary commercial software will naturally turn to open-source alternatives, they say. And by the time those markets become big enough to draw the attention of Microsoft and other commercial software makers, open-source could be as entrenched as Microsoft is in developed countries now.

The article cites localization efforts underway at OpenOffice and presents the argument that since open source development is effectively decoupled from market forces and since open source is, well, open source, you could see grassroots efforts resulting in localization ahead of the growth in demand that would draw the attention of a Microsoft.

Open-source advocates believe they have the upper hand, however. By separating software development from profit motives, they can respond more quickly and completely as computing communities arise.

"It's one of those areas where proprietary software companies are fundamentally at a disadvantage because of their method of allocating resources," said Hiser. "You've got markets that are fragmentary at best, where software as we know it is not economically viable. But that doesn't matter for an open-source project. You just have to have a need and some people willing to work..."

Meanwhile, Microsoft isn't standing still either - see here for MSFT's Office localization for the Indian market.

The article also references another recent Cnet piece on Microsoft's recent decision to segment the Windows and Office product lines by creating an entry-level version to enter the Thai market.

GPL compliance

An important barrier to the broader adoption of open source software by vendors is the complex of issues surrounding intellectual property, compliance with open source licensing terms and the whole-product ramifications of these. And I'm not just referring to the SCO bullshit. This issue will become increasingly important as traditionally proprietary software increasingly intersects with the world of open source and as systems vendors increasingly incorporate open source software components into their products. Look at Linksys as an example - their WiFi AP runs on a Linux platform. Note also that as offshoring/outsourcing increases, these monitoring and compliance costs directly increase.

This Infoworld article references a software startup that recognizes some of these issues and is attempting to address them. Black Duck Software is developing a software package that allows companies to manage and mitigate intellectual property risks associated with reusing software code. It appears to integrate into the product development cycle and allows the vendor to manage compliance with open source licenses in situations where open source components are used.

Black Duck Software develops software solutions that manage and mitigate the intellectual property risks of reusing software code. Black Duck tools automatically detect proprietary and Open Source code, provide OSS license validation and management and ease the software auditing process.

Black Duck Enterprise Edition is an essential tool for Software IP Risk Management. Geared towards multifunctional teams, the Enterprise Edition integrates into the software development cycle from conceptualization, through code development and review, to publishing or distribution, and when software is audited.

Here are some of the challenges and complexity encountered by companies thinking about integrating open source software into their product:
* various different OSS licenses exist (BSD, GPL, etc.), each with unique requirements
* the implications of using or integrating OSS code are not necessarily obvious to software managers or to developers (for instance, if you create a loadable kernel module for Linux from scratch, are you required to contribute it back to the kernel? I've heard differing opinions on this one)
* intellectual property issues around integration of open source and proprietary software
* when development is outsourced (as a lot of it is these days), it becomes harder to monitor or detect whether open source software has crept into the product and what this might imply
* risk of non-compliance. For instance, there was a lot of (negative) online buzz about Linksys and whether they had fully released all their Linux kernel modifications back to the OSS community. Here's more on the Linksys flap:

"This is not really Cisco's fault," Perens explained. "The GPL violation did not originate at Cisco, or Linksys. An off-shore subcontractor supplied Linux as part of the device."... According to Perens, the problem originated with a subcontractor who designed a Linux-based product for Linksys, which was subsequently acquired by Cisco. "Subcontractors in general are not doing enough to inform clients about their obligations under the GPL," Perens said, noting that the problem can be exacerbated when the subcontractor is small, or based off-shore.

Perens says embedded system developers can be especially susceptible to GPL violations. "Often, engineers at smaller companies are left to interpret the GPL, since they may not have lawyers who are conversant with software law as it is excercised in the United States. Contractors will think 'this is embedded, no one can change the source -- so the GPL must not really apply to us.'"

Black Duck just launched and they're going to be at LinuxWorld next week. The management team looks pretty credible and my guess would be that they're going to try to raise venture money. It would be interesting to see how they do. Also, in a similar vein, check out the following (based on some limited googling): MetroWerks' GPL Compliance Toolset.

It occurs to me that what is also missing is a central certification body that a vendor can turn to to certify compliance with various OSS licenses. For WiFi compliance you have the WiFi Alliance, for 802.16 certification you have the WiMax body, for FIPS (Federal Information Product Security) compliance you have a number of certification agencies that will conduct product audits and assessments. I understand FSF does some GPL enforcement but I'm unaware of any structure or organization that assesses and certifies compliance with OSS licenses such as GPL. Anyone have any info on this subject?

Lead users and user innovation networks

Umair recently pointed me to a very interesting paper by Eric von Hippel -'Horizontal innovation networks - by and for users'. I finally got around to reading it and I'll write a post about it soon.

One of my big takeaways from this paper was the idea of lead users – users whose needs significantly anticipate requirements of the broader market well in advance and who create innovation around the existing product to satisfy their unique requirements. This class of users falls outside the traditional technology adoption life cycle taxonomy of Innovators, Early Adopters, Pragmatists and Laggards – in a sense these are Pre-Adopters, adopters in advance of a product. An important distinguishing characteristic of this user group is that they show a propensity to experiment with existing products, tweaking/modding them and often creating new product innovations.

Empirical studies find that innovation by users tends to be concentrated among “lead users” of those products and processes. Lead users are defined as users of a given product or service type that combine two characteristics: (1) lead users expect attractive innovation-related benefits from a solution to their needs and so are motivated to innovate, and (2) lead users experience needs that will become general in a marketplace, but experience them months or years earlier than the majority of the target market (von Hippel 1986). Note that lead users are not the same as early adopters of an innovation. They are typically ahead of the entire adoption curve in that they experience needs before any responsive commercial products exist – and therefore often develop their own solutions.

It is lead users who are behind movements like Linux and open source generally - as has often been noted, OSS developers also tend to be OSS users. von Hippel's point is that the notion of lead users extends beyond just software or high tech products. He finds the same dynamic at work in high-performance windsurfing, for instance.

It pays a firm to cultivate this group as they can serve as a bellwether of trends to come. By creating and tapping into user innovation networks, a firm derives (at least) the following benefits: (1) advance warning of the evolving needs of the broader market, (2) opportunity to leverage innovations stemming from this group and feed them back into the product, (3) ability to leverage this network to maintain competitive lead and stay ahead of the competition.

Creating effective and sticky relationships with this group would entail (1) facilitating the formation and gatherings of communities of such users (through newsgroups, user groups, new product expos), (2) exposing APIs to the product to allow such users to tweak or otherwise customize or modify the product, (3) making early prototypes and alpha versions of future product versions available to these users, (4) maintaining a strong involvement with this community/network to tap into emergent trends as well as innovations, (5) leverage this early warning system as a way to innovate faster than competitors.

Executed well, this strategy can be a powerful source of sustainable competitive advantage through leveraging of exogeneous innovation. Amazon seems to be executing well on such a model through its web services API.

Access to source code

In the years to come, people will identify ways in which the software business changed fundamentally post-Linux. One of the ways it will be seen to have changed is that (non-end-user) buyers will increasingly demand access to source code for modules/components of the product that are purchased from other vendors. One example would be requiring embedded OS vendors to release or at least share their source code.

Anecdotal example: one company I know was considering migrating from an open source-based module that had been under internal development to an externally-supplied substitute. The main reason they rejected the substitute was that it wasn’t open source (or even shared source). What happened was that they started out with an open source version. Over time, they invested effort and resources in improving it to the point where they had it doing what they wanted it to do (in the process improving the set of core competencies that they had in-house). At some point, when the requirements evolved, they were faced with a make-vs-buy decision between staffing to add the new functionality versus buying a third-party software product. The one third-party product that met nearly all the feature requirements was a closed-source, binary-only, application. Having had experience with the open-source version and having had to debug the issues that arose with it, they felt that they really needed to have to have access to the source code for any module they would purchase from a 3rd party – if you have certain competencies in-house, it is much more efficient to identify and remove the software defect yourself than to have to wait for your supplier to fix the problem.

This is just one example. Having access to source code is very powerful:
* greater transparency
* better ability to debug and troubleshoot issues
* ability to make application-specific tweaks to the software module (as an example, a router vendor might value the ability to tweak the OS to improve packet forwarding performance)
* ability to make continual improvements in terms of features and functionality.

While end-user buyers of software are unlikely to care as much about the availability of source code (how many people are really going to want to tinker with their desktop OS or their spreadsheet tool? A vanishingly small fraction), I can see a good justification for systems vendors desiring or even requiring source code from their software suppliers. I would guess that this is the direction in which the embedded software space is going to evolve, which is one reason why WindRiver’s old model (proprietary, closed-source OS) is becoming obsolete and why their recent statements re: Linux are signs of a good move.

Linux in embedded devices

I've been noticing Linux's takeoff in the embedded software space for a while now. Key drivers:
(1) Linux is free
(2) it's good enough (hard real-time capabilities are inessential for many applications, given improvements in processing speeds)
(3) it's open source, which means
(a) adopters of embedded Linux have the freedom/ability to make performance tweaks, etc.
(b) problems/issues are easier to debug and fix
(c) open source developer community is an asset in resolving issues
(4) high level of features and functionality, esp. with regard to the networking stack, for instance.
These were some of the reasons we went with Linux as the embedded OS in our product at my company. Anecdotal evidence, from speaking to other vendors, has confirmed that many are thinking about migrating their products to an embedded Linux base.

Looks like WindRiver (maker of VxWorks) has seen the tide turning and is making a strategic bet on Linux.

Wind River is by far the world's largest embedded software company, and the company does about 30 percent of its business in the telecom and networking markets.

"We have been very aggressive in speaking out against Linux in the past -- in a mode of fighting Linux, saying Linux was bad," says Genard. "That was the wrong fight. We now realize that Linux has a lot of benefit, and has a lot of traction, and can benefit customers. This announcement is about our commitment in recognizing that Linux and VxWorks are both relevant [in the telecom and networking infrastructure markets]," he adds. "Linux is making headway, and we are committed to see how we will support Carrier Grade Linux customers."

Book Watch Plus

Book Watch Plus is a cool service that I discovered a while back. It uses the web services APIs of Amazon, Google and weblogs.com to create a powerful new application. It tracks all blogs that have changed in the last several hours, looks for links to Amazon books in the blog postings and creates a running Top Ten lists of books that are getting a lot of buzz in the blogging community. Using the Amazon and Google web services APIs, it pulls down product information, related news, etc.

Weblogs.com publishes a list of weblogs that have been updated in the last three hours - this list is published as XML and is queryable using XML-RPC or SOAP. Amazon's web services API allows for detailed product information to be pulled down using XML/HTTP or SOAP. Google allows searches to be performed using a SOAP API.

Also see the related Book Watch, Media Watch and AllConsuming and read related articles by Erik Benson and Tim O' Reilly.