Accelerating Software Development with NetApp's P4flex

Accelerating Software Development with NetApp's P4flex

• 1.
  Accelerating Software Development withNetApp's P4flex Bikash Roy Choudhury Narjit Chadha
• 2.
  2 Agenda  Introduction  Whatis p4flex? • Components • Workflows  Perforce in AWS and Hybrid Cloud Model  Key Takeaways  P4flex Demo
• 3.
  NetApp is aglobal provider of software, systems and services to manage and store data © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use3 Annual Revenue Global Employees Patent Assets $6.1B 12,000+ 3,200
• 4.
  Challenges in Applicationdevelopment environment © 2015 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use4 Managing Risk Speed Economics Scale
• 5.
  Faster TTM Dependson.. Developers Focus on • Code & Testing (Unit/Smoke tests) • Maintain Version Control • Continuous Integration • Automate as much as possible Build environments should be • Faster & Scalable • Builds can be repeatable and reliable • Every build should be a release candidate Images by xkcd.com
• 6.
  The equations getssimpler Application Development demands in a DevOps practice Business Impact Value Time Shorter the Time, higher the Value and more Impact with faster Time to Market (TTM) = TTM © 2016 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only By - Bikash Roy Choudhury =
• 7.
  Automating Developers’ environmentwith NetApp using Perforce Helix Continuous Integration (CI) Environment P4 Depot (Source Code) Main Codebase Dev.Branch1 Dev.Branch2 Dev.Branch3 Dev.Branch4 Dev.Branch1 C R L T BUILD CI TEST Baseline1 OK OK Workspace1 (pre-packaged) Developer Environment Snapshot1.1Snapshot Full Builds (Docker Images) • Risk Mitigation • Drive Storage Cost down • Instantaneous Workspace Creation QA/Staging Deploy T R C Snapshot.1 Snapshot1.2 Dev.Branch1.2 BUILD OK Workspace1.2 (pre-packaged) CI TEST OK Baseline1.2 P4 Database © 2016 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only By - Bikash Roy Choudhury P4Flex
• 8.
  P4 Flex Team 8 PMClary Technical Account Manager Lawrence Bunka Sr. Product Strategy Manager Tim Brazil Performance Engineer Paul Allen Senior Integrations Engineer Narjit Chadha Solutions Architect Bikash Roy Choudhury Principal Architect
• 9.
  P4 Flex Overview What is it? • P4 flex is an open-source p4broker script which is shared to the developer community to refine and customized to their own environment. • A starting reference
• 10.
  P4 Flex Overview Support • Support is through the developer community forums in which Perforce and NetApp are members. • Posted on Perforce Workshop • Currently only supports Linux and NFS environments.
• 11.
  P4 Flex Components P4FLEX: • broker.cfg • flex.cfg • flex.py P4 Client p4broker P4D Server P4 Client P4 Client
• 12.
  Requirements  In orderto be able to utilize P4 Flex the following is required: • Python 2.6 or later • P4 Python 2.6 or later APIs • NetApp Manageability Software Development Kit (NMSDK) 5.3.x or later • NetApp RESTful API services will be available in May 2016.
• 13.
  Pre- Requisites  Touse P4 flex, the following are assumed. • SERVER - P4D Server is installed and running - P4Broker installed - LDAP is running or an existence of user accounts and authentication management. - NetApp Cluster Mode Storage Data ONTAP 8.x is used with FlexClone and NFS license enabled • CLIENT: - NFS Client is running on client box - NetApp volume is mounted on client box - The "p4 client" is installed.
• 14.
  Workflow  Create aVolume & add content  Snapshot a Volume  Cloned by a User  Clean up
• 15.
  Workflow Volume snapshot 1snapshot 2 snapshot 3 clone at snapshot 1 clone at snapshot 3 Bob Joe User’s clones created at snapshot on the Volume
• 16.
  Workflow 1. Create aVolume, add content p4 flex volume -s 1G projVolume -Create and Mount the volume at: /p4/projVolume 2. Snapshot the volume p4 volumes p4 flex snapshot -VprojVolume snap1 - Creates a snapshot and hidden Perforce workspace to manage the have list.
• 17.
  Workflow 3. Clone thevolume p4 volumes p4 snapshots p4 flex clone -VprojACE -Ssnap1 myClone -Mounts the clone & places P4CONFIG /p4/myClone .p4config
• 18.
  Workflow  Clean upClones p4 clones p4 clone -d myClone  Clean up Volumes p4 volumes p4 volume -d projACE (snapshots automatically removed)
• 19.
  Perforce Helix inAWS in a Hybrid Cloud Model SnapMirror over VPN / MPLS Metadata Direct Connect Source Code Metadata Source Code Main Datacenter Scalability and Flexibility with NetApp Private Storage (NPS) • Provision on demand resources for bursty workloads • Language and development tools agnostic • Complete data control and location aware • No lock-in with any Service provider or public cloud • Distributed pre-build/build code for QA and system tests NetApp Private Storage (NPS) p4 volumes in (Hybrid) Cloud Perforce in Public Cloud DevOps in Datacenter © 2016 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only By - Bikash Roy Choudhury Pre-Builds Metadata Source Code P4 Server License Server
• 20.
  20 P4 Flex Demonstration
• 21.
  Developer’s advantage usingNetApp technology Claim to fame © 2016 NetApp, Inc. All rights reserved. --- NETAPP CONFIDENTIAL ---21 Scale: 2units=1 minute User WorkSpace CreationTime 72mins Traditional way using “rsync” takes >70minutes Optimized way using NetApp takes <1 min. Scale: 1unit = 1hour Build completion Time Traditional Build times Build times using NetApp reduced by 50% 60TB 3TB Storage Space Efficiency during Code Development, Unit tests and builds Other Storage Up to 40x Storage space reduction using NetApp 24hrs
• 22.
  Thank you! bikash@netapp.com narjit@netapp.com Demo: https://forums.netapp.com/docs/DOC-29884