A Vice President at Amazon Web Services, Amazon’s cloud computing behemoth, published a blog post announcing the release of its own version of Elasticsearch, a powerful open-source software search engine tool.
Elastic is a public company founded in 2012 that is currently worth over $5 billion; the vast majority of its revenue is generated by selling subscription access to Elastic’s search capabilities via the cloud. It’s based in Amsterdam and employs more than 1,200 people.
In the blog post, Adrian Cockcroft, VP of cloud architecture strategy at Amazon Web Services (AWS), explained that the company felt forced to take action because Elastic was “changing the rules” on how its software code could be shared. Those changes, made in the run-up to Elastic’s 2018 IPO, started mixing intellectual property into Elastic’s overall line of software products.
Open-source software is defined as code that can be freely shared and modified by anyone. But now Elastic was telling customers that certain elements in its product mix could not be accessed without payment and that the code could not be freely shared.
Elastic did not explain its strategic shift at the time. But industry observers interpreted the changes as a response to increasing competition from AWS, which had incorporated Elasticsearch’s code and search functionality into its own suite of computing services.
Elastic isn’t the only open source cloud tool company currently looking over its shoulder at AWS. In 2018 alone, at least eight firms have made similar “rule changes” designed to ward off what they see as unfair competition from a company intent on cannibalizing their services.
In his blog post, Cockcroft argued that by making part of its product suite proprietary, Elastic was betraying the core principles of the open source community. “Customers must be able to trust that open source projects stay open,” Cockcroft wrote. “When important open source projects that AWS and our customers depend on begin restricting access, changing licensing terms, or intermingling open source and proprietary software, we will invest to sustain the open source project and community.”
AWS’s announcement did not attract the immediate attention of the Democratic presidential candidates or the growing cadre of antitrust activists who have recently set their sights on Amazon. But in the world of open source and free software, where picayune changes in arcane language can spark the internet equivalent of the Hundred Years War, the release of AWS’s Open Distro for Elasticsearch launched a heated debate.
Open source software has been one of the biggest success stories of the software industry. In 2018 alone, Microsoft’s purchase of the open source software development platform GitHub for $7.5 billion, Salesforce’s purchase of the open source company Mulesoft for $6.5 billion, and IBM’s blockbuster $34 billion purchase of the Linux vendor Red Hat proved that open source is a crucial part of the larger software industry. And there is growing acceptance that the collaborative model of developing open source software is a winning strategy to meet the tech industry’s need for constant innovation. So, when the likes of Amazon start accusing companies of not playing fair, people notice.
Sharone Zitzman, a respected commentator on open source software and the head of developer relations at AppsFlyer, an app development company, called Amazon’s move a “hostile takeover” of Elastic’s business. Steven O’Grady, co-founder of the software industry analyst firm RedMonk, cited it as an example of the “existential threat” that open source companies like Elastic believe a handful of cloud computing giants could pose. Shay Banon, founder and CEO of Elastic, carefully defended Elastic’s new licensing practices, while at the same time making his unhappiness with Amazon crystal clear.
Elastic’s products, Banon wrote, have been “redistributed and rebundled so many times I lost count… There was always a ‘reason’, at times masked with fake altruism or benevolence. None of these have lasted. [Amazon and other vendors] were built to serve their own needs, drive confusion, and splinter the community.”
(AWS declined to comment on the record for this story, and open source companies on the front lines of the confrontation refused to speak in detail about their relationship with Amazon, either providing generic statements or declining interviews.)
The reaction to Amazon’s move wasn’t all negative. Some veterans of the open source community praised Amazon’s defense of open source values, while pointing out the fundamentally messy contradictions of Elastic mixing commercial priorities with open source principles. And fundamentally, adopting open source code is entirely legal.
But the notion that Amazon was presenting itself as an altruistic defender of the digital public commons rankled community veterans like Zitzman, who says that Amazon has a poor reputation for working with the community. (GitHub data shows that Amazon has far fewer employees than Microsoft, Google, or IBM contributing code to open source projects.)
These critics see Amazon’s decision to recreate Elasticsearch as opportunistic . behavior. Amazon, they say, is leveraging its dominant power in cloud computing in order to unfairly reap intellectual property. In doing so, AWS is striking at the Achilles’ heel of open source: lifting the work of others, and renting access to it.
What happened to Elastic, Zitzman says, fits into a “long-standing trend of AWS rolling out managed services of popular open source technology, or replicating such technologies… This move is a text-book commoditization move — providing Elastic’s premium services for free.” Or as Salil Deshpande, a managing director at Bain Capital Ventures and an investor in multiple open source companies, puts it: “It is clear that AWS is using its market power to be anti-competitive.”
On the campaign trail, Senator Elizabeth Warren recently called for the breakup of Amazon, declaring that “you can be an umpire, or you can own a team, but you can’t do both at the same time.” She was referring to Amazon’s role as both an e-commerce platform and a vendor — a scheme that lets the company observe market trends and undercut sellers with in-house products at opportune moments. But Warren’s words might also describe Amazon’s behavior in the open source economy.
If Amazon uses the same ostensibly anticompetitive tactics in the cloud that have helped it establish a commanding position in e-commerce, regulators might want to start paying closer attention. A single company enjoying a dominant position in the cloud could, some critics suggest, result in less overall innovation in the most important part of our digital infrastructure. And that means that the future of antitrust may be up in the cloud.
“What’s happening to open source providers illustrates the raw power that Amazon has,” says Stacy Mitchell, a longtime critic of Amazon and co-director of the Institute for Local Self-Reliance, a nonprofit organization that specializes in challenging “concentrated economic and political power.”
“Amazon’s control of the core infrastructure for exchanging goods and data means they have the ability to set the rules for how everyone else operates,” Mitchell says. “We should recognize that as a kind of governing power.”
AWS is the undisputed goliath of cloud computing, controlling somewhere around 32% of the total worldwide market, or as much as the next three biggest providers combined. When a configuration error knocked large swathes of AWS offline for four hours in February 2017, much of the internet stumbled: Slack, Venmo, BuzzFeed, and even Apple’s iCloud all experienced difficulties.
The list of prominent companies that rely on AWS includes Spotify, Netflix, Airbnb, Comcast, Lyft, Uber, Adobe, and NASA. Big chunks of the U.S. government depend on AWS. Medium, the publisher of this story, uses AWS too. Each of these organizations is taking advantage of the same irresistible value proposition: Off-loading computing operations to the cloud saves money and frees companies to focus their attention elsewhere.
In fact, it might be more revealing to list the companies that don’t use AWS: Walmart, Target, Gap, and Kroger — vendors that are all acutely familiar with Amazon’s cutthroat retail tactics. All of them eschew AWS, Sharone Zitzman says, precisely because “they can’t trust [Amazon] with their intellectual property and business insights.” Walmart goes so far as to require that any technology firms they partner with also eschew using AWS
Last November, Microsoft’s corporate vice president for global retail and consumer goods told CNBC that retailers “want to own their own data, and they want a partner that is not going to be a competitor of theirs in any other part of their businesses.”
The revenue numbers generated by AWS are eye-popping. In the fourth quarter of 2018, AWS accounted for $7.4 billion — or 10% — of Amazon’s revenue, with that number growing a rate of 45% per year. That same quarter, AWS represented 58% of Amazon’s “operating income” — a key measure of profit. By comparison, in 2018, Amazon’s fourth-quarter product sales revenue grew at an annual rate of 8.2%. AWS is increasingly the engine that makes Amazon go.
But “the backbone of the internet” is far more than just a bunch of massive data centers where companies store their information. AWS is both infrastructure and services, meaning that in addition to its low prices, AWS offers a constantly expanding palette of computing essentials, including data processing, analytics, search, security, and integration with multiple programming languages.
The surging growth of cloud services over the past decade has led to a proliferation of companies that sell tools and services that can be integrated easily with AWS, Microsoft’s Azure cloud offering, or Google Cloud. That includes Elastic, but also database firms MongoDB and Redis and the real-time data management company Confluent — all open source software companies operating under a cloud subscription business model referred to in the software industry as SaaS, or “software as a service.”
One of the most striking developments in the evolution of the software industry over the past 20 years has been the emergence of open source companies like Elastic and MongoDB, which have monetized their investment in freely shareable code while harnessing the labor of a large community of programmers. The SaaS subscription model is key to this success story. Customers eagerly outsource the constant labor of updating their software, while open source companies are committed to continually innovating and improving their product. What seemed revolutionary and vaguely anti-capitalist in the late 1990s is now a well-established and thriving business model.
But critics say AWS is endangering that balance by effectively strip-mining the popular open source tools created by others and integrating them into its own cloud product suite. Though some existing licenses mean that that type of behavior is legal, it also makes it much more difficult for companies like Elastic to convince customers to continue paying.
In addition, critics argue that the company’s role as a platform gives it an unfair advantage. Because AWS has thousands of customers, the company has a godlike perspective of broad industry trends, including insight into which third-party tools are most popular. The suspicion, one executive of an open source cloud tool company told me off the record, is that AWS is watching “run rates” — the amount of money spent on a particular tool per year. When they see a service provider like Elastic start to generate serious revenue, Amazon incorporates the functionality of that tool into its own proprietary service. In the words of Warren, that makes AWS both a team owner and an umpire.
Developers like Elastic are starting to take note. “In the last twelve to eighteen months, in fact, a switch has been flipped,” wrote RedMonk analyst Stephen O’Grady in a recent post. “Companies have gone from regarding cloud providers like Amazon, Google or Microsoft as not even worth mentioning as competition to dreadful, existential threat. The fear of these cloud providers has become so overpowering, in fact, that commercial open source vendors have chosen — against counsel, in many cases — to walk down strategic paths that violate open source cultural norms, trigger massive and sustained negative PR and jeopardize relationships with developers, partners and customers.”
Critics like O’Grady argue that Elastic’s recent changes were spurred by a concern of being overshadowed by AWS. Far from defending the principles of open source, critics feel AWS is building its own business by incorporating whatever data management tools turn out to be popular and useful into its own service offering. O’Grady says Elastic isn’t the only open source cloud tool company to shift toward more restrictive licensing agreements over the past year. MongoDB, Redis Labs, and Confluent, among others, have all done the same.
The license changes have caused massive controversy in the open source world, where anything that smacks of proprietary control tends to be treated as apostasy. None of the companies would directly comment on why they made the changes, but there is a common theme that links them all together: An explicit attempt to prevent cloud computing providers, with a special focus on AWS, from taking code that someone else created and delivering it as a service on a cloud platform. The license changes thread a very narrow needle: They aim to maintain the perception that the underlying code is still free to share and modify by the greater community, while preventing specific cloud platforms from taking the functionality represented by that code and offering it as a subscription service.
In other words, open source advocates argue, there’s no problem at all if AWS engineers want to contribute code improvements to their tools, but there absolutely is a problem if AWS wants to start selling its own version of those tools.
Late last summer, Redis, maker of a popular database management tool, changed its licensing terms to prevent AWS from offering Redis functions. For Redis, AWS had become an existential threat. GeekWire’s Tom Krazit wrote last November that “as more and more companies embrace cloud computing and ‘lift and shift’ their existing applications and infrastructure to providers like AWS, it makes a lot of sense [for customers] to use the AWS Redis service… [rather than] a service offered by Redis through the AWS Marketplace.”
The key difference? When someone subscribes to the original Redis via the AWS cloud, Redis gets the fees. When someone uses AWS’s own “Redis service,” AWS gets the money. “This is a problem not just for us but for almost any successful open source project to date,” Ofer Bengal, CEO of Redis, told GeekWire.
Two months later, MongoDB, another extremely successful open source database company, announced its own licensing change. CEO Dev Ittycheria explained that “[w]henever a new open-source project becomes popular, cloud providers strip mine the technology, put the freeware on their platform, capture most if not all of the value but give little back to the community.” (AWS followed up by releasing its own “managed services” version of MongoDB, called DocumentDB, in January 2019.)
Then, in November, at the company’s annual Reinvent conference, AWS announced a service called Managed Streaming for Kafka, which duplicated the functionality of an open source tool originally built by the event-streaming platform developer Confluent. Two weeks later, Confluent announced its own license changes.
Kreps declined an interview request, but it was clear who he was referring to. Though retweets are not always endorsements, when Elastic CEO Shay Banon tweeted a link to his post defending Elastic’s license changes, Kreps promptly retweeted it. The battle lines seem to be drawn. On one side, the commercial open source cloud tool providers who feel the ground shifting underneath them, and on the other, an earthquake named AWS.
The most recent confirmation of the split between AWS and the open source world came at the Google Cloud Next conference in mid-April. Google announced a series of partnerships with open source companies—notably, many of the firms that changed their licensing terms in 2018, including MongoDB, Redis, Confluent, and Elastic. Google Cloud CEO Thomas Kurian told TechCrunch, “Any platform that wins in the end is always about fostering rather than shutting down an ecosystem… In order to sustain the company behind the open-source technology, they need a monetization vehicle. If the cloud provider attacks them and takes that away, then they are not viable and it deteriorates the open-source community.”
Last December, Scott Galloway, a professor of marketing at New York University, predicted that, in 2019, Amazon would spin off AWS into its own company, in part, reported BusinessInsider, to “placate regulators who are starting to scrutinize [Amazon’s] anticompetitive practices.”
Some critics of Amazon supported the idea, on the simple theory that cutting off the company’s retail operation from its cloud computing money tree would force Amazon to stop unfairly subsidizing its e-commerce operations and undercutting the competition.
But while splitting off AWS from Amazon might weaken Amazon as an overall entity, it would do nothing to address the fact that AWS itself may be big enough to trigger its own antitrust action. And it seems as if AWS has already begun acting like a monopoly power when it comes to the open source community.
Salil Deshpande, the Bain Capital Ventures managing director, says that Amazon has three behaviors toward open source that are offensive and aggressive: “They take open source code produced by others, run it as a commercial service, and give nothing back to the commercial entity that produces and maintains the open source, thereby intercepting the monetization of the open source; they fork projects and forcibly wrestle control away from the commercial entity that produces and maintains the open source projects, as they did in the case of Elasticsearch; and they hijack open source APIs and place them on top of their own proprietary solutions, thereby siphoning off customers from the open source project to their own proprietary solution.”
AWS’s 30% share of cloud computing, as well as its astronomic growth, should be cause for national concern. Antitrust focus on retail is well-founded: Retail is a huge part of the current U.S. economy. But the infrastructure of online computing is just as obviously a huge part of the future of the nation’s economy.
Hal Singer, an antitrust economist at George Washington University, says one of the primary problems with Amazon’s economic dominance is that if one company overwhelmingly controls a market, there is less and less room for what he calls “edge innovation” — new products and services created through the competitive pressures. And even though a strong case can be made that right now the world of open source software is thriving, observers of the clash between AWS and open source worry that the room for further innovation may be rapidly shrinking.
Elastic’s current stock price, as of April 23, was over $10 higher than on the day of its IPO, which suggests the company is doing reasonably well in the short term. But the disconcerting scenario is that AWS’s introduction of Open Distro for Elasticsearch might gut Elastic’s ability to continue generating revenue from its own version of its search tool in the long term. And that, in turn, will inevitably repress its capability for making future innovations — or, in the worst-case scenario, result in Elastic’s ultimate demise. The e-commerce world, after all, is all too familiar with the destructive force Amazon can wreak on almost any market it enters.
“This has the potential to have very destructive fallout for the open source world,” Zitzman says. “When a corporate giant like AWS doesn’t play fair, they are actually shooting themselves in the foot, as they will eventually find themselves without open technologies to roll out.”
