Historical Context¶
The NSF high performance computing (HPC) initiative in the 1980s provided an early glimpse of the utility computing vision, though limited to the HPC community. However, the advent of personal computers delayed this vision by putting the focus on smaller-scale computing available to all, rather than high-performance computing for a smaller community. The personal computer industry was started by hobbyists who wanted their own machines to tinker with, and was fuelled by Moore’s Law that enabled personal computers to keep up rapidly increasing user demands.
美国国家科学基金会(NSF)在20世纪80年代的高性能计算(HPC)计划为实用计算的愿景提供了早期的预览,尽管其仅限于HPC社区。然而,个人计算机的出现推迟了这一愿景的实现,将重点放在面向大众的小规模计算上,而不是面向小众社区的高性能计算。个人计算机行业最初由希望拥有自己机器以进行研究的业余爱好者发起,并受到摩尔定律的推动,摩尔定律使个人计算机能够迅速满足日益增长的用户需求。
The emergence of the Internet led quickly to several popular services being accessible worldwide, including e-mail, bulletin board systems, and games. The advent of the World Wide Web ignited an explosion of new services such as search, on-line retail, and eventually social media. To cope with the resulting scale, these services had to build datacenters and design their own complex distributed systems. This path was followed by many companies – such as Yahoo!, Google, eBay, and Amazon – but the onus of creating large-scale service-specific infrastructures became a barrier to entry in the Internet services market because these steps required huge investments which most could not afford.
互联网的兴起迅速使多种流行服务在全球范围内可访问,包括电子邮件、公告板系统和游戏。万维网的出现引发了新服务的爆发式增长,如搜索、在线零售,最终还有社交媒体。为了应对随之而来的规模问题,这些服务不得不建设数据中心并设计自己复杂的分布式系统。许多公司都走上了这条道路——如Yahoo!、Google、eBay和Amazon——但创建大规模特定服务基础设施的负担成为了进入互联网服务市场的障碍,因为这些步骤需要巨额投资,大多数企业难以承担。
This changed in 2006 when Amazon launched S3 and EC2, kicking off the age of cloud computing by democratizing access to compute/storage and promoting the “pay-as-you-go” business model. This, together with the end of Moore’s Law (which made it quite expensive to build and scale services in on-premise clusters), led to a renewed push towards building what could have become utility computing. However, commercial trends have pushed us in a different direction.
这一局面在2006年发生了变化,当时Amazon推出了S3和EC2,开启了云计算的时代,民主化了对计算和存储资源的访问,并推广了“按需付费”的商业模式。这一变化,加上摩尔定律的终结(使得在本地集群中构建和扩展服务变得相当昂贵),推动了构建本可以成为实用计算的重新努力。然而,商业趋势将我们推向了另一个方向。
In the early years when Amazon was dominant in the cloud, they set the de facto standard for cloud computing. However, in the past decade several competitors have emerged in this market. According to a recent report [6], AWS now owns just 32% of the market, followed by Microsoft with 19%, Google with 7%, Alibaba with 6%, and the other clouds (such as IBM and Oracle) splitting the rest of the remaining 37% of the cloud market (due to roundoff errors this adds up to 101%). This competition has led to lower prices and an increasing array of products and services. For instance, AWS alone offers more than 175 products and services [8]. However, many of these services are proprietary, and these proprietary services are one of the main ways cloud providers differentiate themselves. For example, each cloud has its own APIs for managing clusters, its own version of object store, its own version of a data warehouse, its own serverless offering, and so on. Applications developed on one cloud often do not work on a different cloud without substantial changes, the same way an application written for Microsoft Windows requires substantial changes to work on Mac OS. Thus, competition in the cloud market has led us away from the vision of utility computing.
在亚马逊主导云计算的早期,他们设立了云计算的事实标准。然而,在过去十年中,几家竞争者涌入了这一市场。根据最近的一份报告【6】,AWS 现在占有云市场的32%,其次是微软的19%、谷歌的7%、阿里巴巴的6%,其他云服务(如IBM和Oracle)瓜分了剩下的37%(由于四舍五入误差,总和为101%)。这种竞争导致价格下降以及产品和服务种类的增加。例如,AWS 单独就提供了超过175种产品和服务【8】。然而,其中许多服务都是专有的,而这些专有服务是云服务提供商区分彼此的主要方式之一。比如,每个云都有自己的集群管理API、自己的对象存储版本、自己的数据仓库版本、自己的无服务器计算产品,等等。在一个云上开发的应用程序往往不能在另一个云上正常运行,除非进行大量更改,就像为Microsoft Windows编写的应用程序需要进行大量修改才能在Mac OS上运行一样。因此,云市场的竞争将我们引向了远离实用计算的愿景。
Of course there have been many calls for standardizing cloud computing [28, 39], but these have had little impact on the increasing drive towards cloud differentiation. The business models of current cloud providers are built around attracting, and then retaining, customers, and this goes against offering a purely commoditized service. The question we address, then, is how do we make progress towards the goal of utility computing?
当然,关于云计算标准化的呼声由来已久【28, 39】,但这些呼吁对云计算日益趋向差异化的趋势几乎没有产生影响。当前云服务提供商的商业模式围绕吸引并留住客户而构建,这与提供纯粹商品化的服务背道而驰。因此,我们要探讨的问题是,如何朝着实用计算的目标迈进?
Our proposal is called Sky computing, to connote that we are trying to look beyond individual clouds. However, we are not the first to use the term "Sky computing”. Starting in 2009, several papers have proposed designs with this name [30, 34, 35]. However, these papers focus on particular technical solutions, such as running middleware (e.g., Nimbus) on a cross-cloud Infrastructure-as-a-Service platform, and target specific workloads such as high-performance computing (HPC). This paper takes a broader view of Sky computing, proposing it as the universal software platform for future applications and considering how technical trends and the market forces can play a critical role in the emergence of Sky computing.
我们提出的概念称为“天空计算”(Sky computing),意在表明我们试图超越单个云的视角。然而,我们并不是第一个使用“天空计算”这一术语的。从2009年开始,已有几篇论文提出了这一名称下的设计【30, 34, 35】。不过,这些论文主要集中于特定的技术解决方案,如在跨云基础的设施即服务平台(IaaS)上运行中间件(例如Nimbus),并针对高性能计算(HPC)等特定工作负载。本文则采取了更广泛的视角,将“天空计算”作为未来应用程序的通用软件平台,并探讨技术趋势和市场力量在“天空计算”兴起过程中如何发挥关键作用。