Downloaded 12 times
Uploaded byPatrick Chanezon
Using Open Source and Open Standards in the Platform game
This document discusses Docker Inc.'s efforts to establish an open platform for containers through open source and open standards. It summarizes Docker's work on projects like runc, OCI, and containerd that established the building blocks for container technologies. It also notes Docker's rapid growth from 2013 to 2016 and transition from focusing initially on developers to later prioritizing orchestration and production usage through projects like Docker Swarm. The document evaluates how Docker's open source strategy helped establish it as the dominant player in the container space initially before other companies began competing in the container platform market.
Using Open Source and Open Standards in the Platform game
- 1. Patrick Chanezon Docker Inc. @chanezon UsingOpen Source and Open Standards in the Platform game. War stories and lessons learned
- 2. French Polyglot Platforms Software Plumber San Francisco DeveloperRelations @chanezon
- 3. What is aPlatform • two-side markets: enables transactions between 2 sides • Asymmetry: one side pays more than the other • Network Effects: value grows as N^2 • cross side effects • same side effects
- 4. Two-sided markets examples 4 Eisenmann,Thomas R. and Parker, Geoffrey and Van Alstyne, Marshall W., Strategies for Two Sided Markets (October 1, 2006). Harvard Business Review, Vol. October, 2006.
- 5. Successful two-sided softwarePlatforms in software 5
- 6.
- 7. Two-sided markets definition 7 Jean-CharlesRochet, and Jean Tirole, “Two-Sided Markets: A Progress Report”, The RAND Journal of Economics, vol. 35, n. 3, 2006, pp. 645–667
- 8. When is amarket not two-sided? 8 • When the 2 sides can transact directly without the platform
- 9. Platform Strategies Control Openness CompatibilityControlled migration Open migration Performance Performance play Discontinuity Information Rules, Shapiro & Varian
- 10. Sun Portal Serverand JSR 168 (2002)
- 11. Horizontal Portal Marketin 2003
- 13. Portal Platform Enterprise DevelopersEnterprise IT Portal Platfor m
- 14. Open Standards: JSR168 Fragmentation: many Proprietary APIs Enterprise adoption Sun ONE bundle: all Enterprise servers in one integrated product Create a standard: Sun & IBM Goal: get more developers, a rising tide lifts all boats
- 15.
- 16. Open Source ReferenceImplementation
- 17.
- 18.
- 19.
- 20. Evaluation JSR 168 createdan ecosystem of portlet developers Created value for Enterprise developers and Enterprise IT departments Created value for shareholders: Oracle bought Sun & BEA, #3 after IBM Microsoft succeeded without it Stopped being relevant with Ajax, Google Gadgets, OpenSocial, OAuth and APIs.
- 21.
- 22.
- 23.
- 24. Goal • Allow largedirect customers to automate Ads creation and management • Ensure intermediary cross platform ads platforms expose all Adwords functionalities Open Source client libraries Strict terms and conditions • Required Minimum Functionality • API usage Fees • Update every 2 months, forced to update after 3 releases Google AdWords API
- 25.
- 26. Google AdWords APIClient Libraries
- 27. Google AdWords APIClient Libraries
- 28. Cross Network AdsPlatforms
- 29. Pretty successful No needfor Open Standards Minimal use of Open Source Adwords API touches a significant % of all spend on Adwords Evaluation
- 30.
- 31. Social Developer Platforms UsersDevelopers Social Platfor m
- 32.
- 33.
- 34.
- 35.
- 36. Facebook created APIs,many apps Facebook growing 50% yoy, but still mainly US Social networks are fragmented by Geo Fear that in the long term social networking would eat Google’s Ads share Idea: create an open standard for social applications, adopted by all social networks worldwide, then provide monetization on top Google has been very successful at monetizing an open system like the web Facebook was a threat to that in social Social Networks in 2007
- 37.
- 38.
- 39.
- 40.
- 41.
- 42.
- 43.
- 44.
- 45.
- 46.
- 47.
- 49. Total Failure OpenSocial isan excellent case study of failing to establish a successful platform relying in majority on open source and open standards Wrong Map! • Simon Wardley says: look at the Map! • We were not looking at the right Map! • We played our Go game at the wrong layer: the real fight was at the UI layer, not the API layer, four on Users first, not Developers Standardizing worked for the tech aspects …but lacked unified app store and monetization Evaluation
- 50.
- 51.
- 52. 2000 HTML 4.01 2004W3C working on XHTML, going nowhere 2004 WHATWG 2008 HTML5 First public working draft 2D, audio, video, storage, workers,… Against Flash, Silverlight, Applets, ActiveX, native mobile apps, … Specs and Working code: Mozilla, Opera, Chrome Google great at monetizing an open ecosystem HTML5, Webkit and Chrome
- 53. Worked for theweb Did not work for mobile… but Android worked there Evaluation
- 54.
- 55. App Engine Platform DevelopersIT Departments GAE Platfor m
- 56. Cloud Stack -Classic Pyramid Platform As A Service Infrastructure As A Service Software As A Service
- 57. AWS Created EC2in 2006: IaaS 2008 App Engine created the Paas category No standards OSS only for desktop SDK Initially only Python, then Java GAE for business in 2010 Google App Engine
- 58. Failure Never took offbeyond large scale consumer apps Too many limitations Enterprises wanted hybrid cloud Google Cloud Platform rebooted in 2011 with Google Compute Engine Platforms like BigQuery were more successful Yesterday, Spanner for productized! Evaluation
- 59.
- 60. Cloud Foundry Platform DevelopersIT Departments Cloud Foundry Platfor m
- 61. 2011: PaaS iswhere the value is for developers to become Agile Enterprises want Hybrid Avoid lock-in: open source PaaS Cloud Foundry
- 62. Cloud Stack -Classic Pyramid Platform As A Service Infrastructure As A Service Software As A Service
- 63. Cloud Stack -By Value to developers Infrastructure As A Service Software As A Service Platform As A Service
- 64. Main Risk: Lock-In 66 Welcometo the hotel california Such a lovely place Such a lovely face Plenty of room at the hotel california Any time of year, you can find it here Last thing I remember, I was Running for the door I had to find the passage back To the place I was before ’relax,’ said the night man, We are programmed to receive. You can checkout any time you like, But you can never leave!
- 65. Cloud Foundry: TheOpen PaaS 67 Private Clouds Public Clouds Micro Clouds Data Services Other Services Msg Services vFabric Postgres vFabric RabbitMQTM • Open Source: Apache 2 Licensed • multi language/frameworks • multi services • multi cloud
- 66.
- 67. 69 500 – 5,000VMs 40+ unique node types 75+ unique software packages 75+ unique environments 2x/week cf.com updates 24x7x365 non-stop operation No-downtime deployments Reliable, robust, repeatable deployments, updates, capacity adjustments Small teams manage many instances Production Grade Cloud Foundry Clusters Google style problem Google style solution cloudfoundry.com production, staging, stress, qa, dev
- 68. 70 BOSH: under thehood bosh cli redis natsdb director healthmon IaaS CPI workers stemcell agent blobs “BOSH is deployed by BOSH” cloudfoundry.com BOSH User active jobs disk
- 69. Semi success Spun offby VMware with all middleware to Pivotal Complex architecture, complex to operate: BOSH Made less relevant by containers “still a under 1b$ market” Charles Fitzgerald Evaluation
- 70.
- 71. Microsoft Azure Platform DevelopersIT Departments Azure Platfor m
- 72. IaaS focus Hybrid strategy Own.NET developers Goal: Attract Linux workloads, Java & OSS developers Leverage trends in the community: Agile, Devops Microsoft Azure
- 73. Cloud Stack –Where the fight is in 2013 Infrastructure As A Service Software As A Service Platform As A Service
- 74.
- 75.
- 76.
- 77. Major datacenter CDN node Livesub-region Announced sub-region Partner-operated sub-region 24 x 7 x 365 support. 89 markets worldwide. 2x Compute and storage every six months.
- 79. Devops • Cultural movement •Inspired by agile methods • People, Processes & Tools • Continuous delivery • Infrastructure as code • Cross silo collaboration • Small iterations • Feedback loop, measurement Image from Patrick Debois http://www.slideshare.net/jedi4ever/devops-the-war-is-over-if-you-want-it http://www.slideshare.net/jedi4ever/devopsdays-downundervfinal
- 80. •Consumer software isbecoming like fashion •Phone apps, social apps •Short lifetime, Fast lifecycles •A/B testing •Enterprise •Clay Shirky situational apps Kent Beck, Usenix 2011 Talk, “Software G-Forces” Agility as a survival skill
- 81. Cloud Platforms enableAgile • Built-in Scalability • Faster Iterations • Focus on Design • Focus on Business Value
- 82.
- 83.
- 84.
- 85. Behavior Driven Development/ Infra
- 86. DevOps: the threestage conversation 3 Products2 Process1 People
- 87.
- 88. Open source ecosystem GitHub Codeplex Selenium GradleGrunt HudsonJenkins Configuration Gradle Chef Grunt Hudson Travis Jenkins Puppet Labs Vagrant Fog Alerting Monitor CactiZabbix RedmineJIRA Public Cloud On-premises Microsoft Azure Linux Puppet Labs Eclipse Visual Studio
- 89. MSOpenTech & DevOps Interoperabilitythrough open source and open standards Wholly owned subsidiary of Microsoft Independent cadence enables a closer interaction with open source partners Team of high skilled executives http://msopentech.com/blog/project-categories/devops/
- 90. Microsoft ecosystem Team FoundationServer Team Foundation Server Microsoft Test Manager Microsoft Monitoring Agent Service Manager PowerShell xPlat CLI WAML Resource Groups Visual Studio Online Application Insights Team Foundation Server Release Management for Visual Studio Release Management for Visual Studio ONE CONSISTENT PLATFORM ON-PREMISES SERVICE PROVIDER Microsoft Azure Release Management for Visual Studio System Center Operations Manager OR TFVC GIT Automation Service
- 92. Developer Platform Evangelism •Strategic Engagement team • We build Epic stuff with customers • Architecture Design Session • Hackfest: code together • What can we do together? • When do you want to start? • patrick@chanezon.com
- 93. Azure is agreat platform for Java Developers
- 94. Azure Execution Models VMs CloudServices Websites
- 95.
- 96.
- 97.
- 98.
- 99. Success: 28% marketshare in Cloud Contribute to many open source communities, and standards Focus on trends and projects/products behind them: Agile, Devops, Microservices Massive adoption of Docker / Containers Evaluation
- 100. Docker - OCI- containerd (2015-2017)
- 101.
- 102. BUILD Development Environments SHIP Registry: SecureContent & Collaboration RUN Control Plane: Deploy, Orchestrate, Manage, Scale Networking Volumes MonitoringLoggingConfig MgtCI/CD IT Operations Developers IT Operations Docker Platform
- 103. 5 Goldilocks and the3 XaaS Just rightToo highToo low IaaS PaaS CaaS
- 104. 5 Goldilocks and the3 XaaS Platform As A Service Infrastructure As A Service Software As A Service Too high Too low Just right Container As A Service
- 105.
- 106. Patrick Chanezon Docker Inc. @chanezon chanezon@docker.com DockerOrchestration Welcome to the Jungle! 2015
- 107. DockerCon 2015: runc,OCI December 2016: containerd Docker
- 108.
- 109. 2013-05 2013-06 2013-07 2013-08 2013-09 2013-10 2013-11 2013-12 2014-01 1,000,000 0 2014-02 2014-03 2014-04 2014-05 2014-06 2014-07 2014-08 2014-09 2014-10 2014-11 2014-12 2015-01 2015-02 2015-03 2015-04 2015-05 2015-06 2015-07 2015-08 2015-09 2015-10 2015-11 2015-12 2016-01 ~ 2016-09 1,000,000,000 ~ 10,000,000 9,000,000 8,000,000 7,000,000 6,000,000 5,000,000 4,000,000 3,000,000 2,000,000 6,000,000,000 5,750,000,000 5,500,000,000 5,250,000,000 5,000,000,000 4,750,000,000 4.500,000,000 4,250,000,000 4,000,000,000 3,750,000,000 3,500,000,000 3,250,000,000 3,000,000,000 2,750,000,000 2,500,000,000 2,250,000,000 2,000,000,000 1,750,000,000 1,500,000,000 1,250,000,000 Notary runC • containerd • HyperKit, VPNKit, DataKit • SwarmKit • libcontainer • libnetwork • • Docker 1.8 : Docker Content Trust • Docker for Mac Docker for Windows • Docker 1.12 with built-in orchestration • Docker 0.9 : Pluggable execution • Docker 1.7 : Multi-Host Networking • Docker 1.11: OCI support
- 110. 112 User first, plumbinglater Obsess over your user’s problems. Spin out open components along the way. React, GraphQL… Webkit, Swift, LLVM… Chromium, Tensorflow… Hystrix, Hasgard… containerd, notary, runC, hyperkit…
- 111. containerd’s role inthe container ecosystem
- 112.
- 113.
- 114. Containerd 1.0 inDocker (Q2, 2017)
- 115. A commercial product, builton a development platform, built on infrastructure, built on standards. Docker is building a stack to program the Internet
- 116. Docker = ContainerPlatform for Complete Distributed App Lifecycle 118 Docker Differentiators SwarmKit Containerd InfraKit Container Orchestration Core Container Runtime Infrastructure Docker API Docker CLI Docker Compose Docker Build Auth Docker Content Trust Docker Distribution Plugins Storage Networking Docker Commercial Platform Docker Datacenter
- 117. Docker Containers asa Service platform 119 BUILD Developer Workflows SHIP Registry Services RUN Management Docker for Mac and Windows Docker Trusted Registry Docker Universal Control Plane Docker Cloud Docker Container Engine Ecosystem Plugins and Integrations
- 118. Ongoing but appearsto be successful OCI adopted by everyone containerd welcomed by top 3 cloud providers Clear difference between open source plumbing and standards and Docker platform we can monetize Early days, join the adventure! OCI, containerd Evaluation
- 119.
- 120. Platform Strategies Control Openness CompatibilityControlled migration Open migration Performance Performance play Discontinuity Information Rules, Shapiro & Varian
- 121. Platform Strategies Control Openness CompatibilityAzure JSR 168, Open Social, HTML5, Docker Performance AdWords API, Google App Engine Cloud Foundry Information Rules, Shapiro & Varian
- 122. • Open Sourcecan help build one side of your platform: JSR 168, Cloud Foundry, Azure … but you need to provide monetization opportunities (Open Social failed, Azure succeeded) • Open Source is not always useful: Facebook, AdWords • Open Source does not always win against proprietary platforms: OpenSocial • Open Standards work when they cover an area where innovation slows down. They help grow the market for everyone • true: JSR 168, Docker/OCI • false: OpenSocial, too early Lessons learned
- 123. • In afast moving market, watch out for brutal changes • JSR168/WSRP -> Ajax, Gadgets; HTML5 -> Mobile/Android; Cloud Foundry -> containers • Engage at the right level in the stack (timing is of the essence) • App Engine PaaS when IaaS mattered • OpenSocial API when UI mattered • Open Standards should be small and solve one problem well • Example: OCI • Counter example: WSRP (WS-*) • Open Source success is all about the community • Azure engaging with the right communities • Docker Community, OCI, containerd Lessons learned
- 124. • Book: CarlShapiro, Hal R. Varian (1999). Information Rules: A Strategic Guide to the Network Economy • Carl Shapiro, Hal R. Varian (1999). Art of Standards Wars • Jean-Charles Rochet and Jean Tirole (2004). Two-Sided Markets: An Overview∗ • Book: Invisible Engines : How Software Platforms Drive Innovation and Transform Industries – David Evans, Andrei Hagiu, and Richard Schmalensee (2006). • Thomas Eisenmann, Geoffrey Parker, and Marshall Van Alstyne (2006). "Strategies for Two-Sided Markets." Harvard Business Review. • Nicholas Economides and Evangelos Katsamakas “Two-Sided Competition of Proprietary vs. Open Source Technology Platforms and the Implications for the Software Industry” (2006) Management Science 200652:7 , 1057-1071 • Thomas Eisenmann, Geoffrey Parker, Marshall Van Alstyne “Platform Envelopment” (2010) Bibliography
- 125. • Slides • http://www.slideshare.net/chanezon •Cover illustration from “The Meijin's Retirement Game”, John Fairbairn • Questions • @chanezon Resources
- 126.
Editor's Notes
- #5 OCI describes the words in your json file Json file asks runC Takes a root file system and creates an isolated process Runtime executor -- runc
- #6 OCI describes the words in your json file Json file asks runC Takes a root file system and creates an isolated process Runtime executor -- runc
- #8 OCI describes the words in your json file Json file asks runC Takes a root file system and creates an isolated process Runtime executor -- runc
- #9 OCI describes the words in your json file Json file asks runC Takes a root file system and creates an isolated process Runtime executor -- runc
- #77 On-premises AND Cloud: We believe in a world where you’re integrating public cloud with your on-premises infrastructure, and using each where it makes sense, in conjunction with each other. Think and, not or. It’s not an on-premises OR cloud proposition – it’s an AND proposition. And when we say integration, we mean true integration – across infrastructure, apps, identity, and databases. This is what we call hybrid. Microsoft is the only company which has the necessary assets across virtualization, identity, data platform , development and management to provide a consistent experiences across on-premises, our cloud and 3rd party service providers. This vision and strategy - called “Cloud OS” – is what we aim to deliver for our customers. If you choose look at other Cloud vendors that provide public OR private cloud offerings (Amazon, VMware, or Google), you have to cobble together disparate offerings and you will not get a seamless experience.
- #81 You can use this to explain all the Azure building blocks – the compute services at the top and then the other blocks of services – data, networking, Apps…. OR you can go with the next set of slides… YOUR CHOICE…
- #89 People = Culture Fundamental attributes of successful cultures: Shared mission and incentives: infrastructure as code, apps as services, DevOps/all as teams You need to consider your hardware as a commodity, (don't give your servers names) , servers are like farm animals, it is just harder if you let theids name them Build deep instrumentation into services, push complexity up the stack Rally around agile, shared metrics, CI, service owners on call, etc. Changing the culture: any change takes time, changing culture is no exception and you can't do it alone, exploit compelling events to change culture: downtimes, cloud adoption, devops buzz PROCESS Definition and design, compliance, and continuous improvement PEOPLE Responsibilities, management, skills development, and discipline Products Tools and infrastructure
- #92 Timing: 2 minutes To further advance the company’s long-standing investments in openness including interoperability, open standards and open source, Microsoft launched a wholly-owned subsidiary Microsoft Open Technologies, Inc. (MS Open Tech) in early 2012. We are motivated by the core belief that open technology is a powerful enabler – and this concept underscores all of the work we do to create technical bridges between Microsoft and non-Microsoft technologies. We are an organization of engineers, standards professionals and technical evangelists who are both experienced in and passionate about working with an equally diverse set of technologies. In addition, we leverage our ability to marshal engineering talent from Microsoft on a project basis through the MS Open Tech Hub engineering program to help facilitate the exchange and evolution of open source engineering best practices. Code talks within MS Open Tech. Many of our primary activities encompass building open source code and promoting the development and adoption of open technical standards specifications to deliver a more seamless experience across hardware, software and devices. Please visit our Projects page for more details about our community contributions in these areas Main Executives MS Open Tech Executives Jean Paoli President In his role as President of MS Open Tech, Jean leads a diverse team of engineers, standards professionals and technical evangelists to promote open platform development and customer choice by delivering new technologies in collaboration with open source and standards communities. A passionate advocate of open standards since 1985, Jean was one of the co-creators of the XML 1.0 standard via the World Wide Web Consortium (W3C), and he has garnered multiple industry awards for his work. Upon joining Microsoft Corporation in 1996, Jean jump-started XML development and managed the team that delivered msxml, the software that XML-enabled both Internet Explorer and the Windows operating system. Jean helped architect Office XML support and was instrumental in creating InfoPath, the XML Office Electronic Forms application. He also participated in ISO/IEC SC34/ WG4 and as co-chair of the TC45 Ecma standards committee that formalized the Office Open XML Format as an international standard. Operating as a distinct business operation since 2012, Jean’s team at MS Open Tech has worked closely with many business groups across Microsoft to promote several technical standards, including W3C’s HTML5, IETF’s HTTP 2.0, Cloud standards in DMTF and OASIS. The team also collaborates with a broad variety of development communities to contribute tools that promote interoperability between Microsoft technologies within open source environments such as Node.js, MongoDB and Phonegap. Prior to Microsoft, Jean worked with a number of European research institutes, including INRIA in France, where he designed systems to facilitate data exchange for major corporations Gianugo Rabellino Senior Director of Open Source Communities With more than 20 years of experience in the open source community, Gianugo brings a deep understanding of open technologies and platforms to his role as Senior Director of Open Source Communities at MS Open Tech. He is charged with promoting the team’s broad engagement with developer communities to help create new business opportunities using Microsoft and non-Microsoft technologies. Gianugo has also been an active member of the Apache Software Foundation since 1999, where he currently serves as vice president of the Apache XML Project Management Committee. Additionally, he assists on a number of projects as a mentor through the Apache Incubator, and speaks at conferences around the world about open development. Previously, Gianugo was the founder and CEO of Sourcesense, the leading open source services company in Europe, where he drove sustained double-digit growth to expand its operations across Italy, the Netherlands and the UK. Gianugo has also held a variety of senior management roles at Pro-netics, Ksolutions, and Bibop Research where he was responsible for the software development and system/network administration groups and worked with several customers including Sun, IBM, Oracle, ISP and the Apache Software Foundation. As an open source technical and policy consultant, he co-founded the first official Linux association in Italy, elevating Linux and open source development to the mainstream within that region. He received his undergraduate degree from Liceo Classico Gabriello Chiabrera and his graduate degree from Universita degli Studi di Genova. Twitter: @Gianugo Kamaljit Bath Director of Engineering Team Kamaljit joined MS Open Tech with nearly 20 years of diverse software industry experience at various levels. He leads the company’s engineering team to create standards-based tools that facilitate interoperability between open source and Microsoft products and technologies, which has resulted in open source project contributions such as: WebKit, Blink, Node.js, Apache QPID, jQuery and Apache Cordova. Kamaljit also coordinates the Interoperability Executive Customer (IEC) Council – an advisory board comprised of ~35 CIOs representing global public and private sector enterprises. In this capacity, he works closely with many Microsoft product teams, standards and policy teams, as well as the Microsoft Trustworthy Computing and Engineering Excellence teams, to strategize on features, best practices and trainings that align with the objective of achieving greater interoperability with Microsoft products and technologies. Previously, Kamaljit spearheaded Microsoft’s first-ever participation in an Apache-sponsored open source project, managing the Apache Stonehenge incubator to showcase the interoperability of web services standards. He was also lead program manager on both the Microsoft Office InfoPath and Microsoft SQL Server teams. Prior to Microsoft, he worked as an Oracle database and forms programmer and mainframe to client-server systems analyst in several Fortune 500 companies. Kamaljit received his Bachelor of Science in Computer Science from National Institute of Technology, Allahabad, India in India. Paul Cotton Partner Group Manager Paul leads the standards team MS Open Tech. He has nearly 40 years of experience in all aspects of software development. He is credited with Microsoft’s cloud computing interoperability and standards strategy and he previously led the company’s multi-year web services standardization efforts within W3C, OASIS and WS-I. After several leadership roles within the W3C, Paul presently serves as co-chair of the working group responsible for the HTML5 specification. Paul is also a Microsoft Standards Advisor supporting cross-divisional strategic standards issues and acts chair of Canadian Advisory Committees for the International Organization for Standardization (ISO) - SC 38 Cloud Computing and SC 34 Document Description and Processing Languages. Paul also architected, developed and managed the SQL-based full-text product with an Open DataBase Connectivity (ODBC) interface, and was a major contributor to consortium efforts such as ATA SFQL, SQL Access Group CLI, SQLJ and SQLX. Prior to Microsoft, Paul founded a consulting company and software vendor, Officesmiths, where he was an architect and development manager for a successful office automation software product. He has served as the United Nations advisor and project manager to successful software projects in Chile and Burma. He has also worked for IBM Canada, Fulcrum Technologies, PBC & Associates, Alphatext Inc., and Statistics Canada. Paul received his undergraduate degree, and a Masters of Mathematics from the University of Waterloo.
- #105 Local development environments Self service app images Build, Test, Deploy applications Define app behavior and infra needs Registry services for image storage, management and distribution IT Ops maintains library of secure base content Manage role based access to repos/images Management consoles Provision, manage infrastructure resources Monitor, manage, scale infrastructure and applications