Downloaded 11 times
Uploaded byKellyn Pot'Vin-Gorman
Oracle Open World 2017 Delphix and DBVisit
The document discusses challenges with moving databases to the cloud and proposes a solution using data virtualization. It summarizes that virtualizing databases with tools like Delphix and DBVisit allows for instant provisioning of development environments without physical copies. Databases are packaged into "data pods" that can be easily replicated and kept in sync. This streamlines cloud migrations by removing bottlenecks around copying and moving large amounts of database data.
Oracle Open World 2017 Delphix and DBVisit
- 1. © 2017 Delphix.All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential. Kellyn Pot’Vin-Gorman | Technical Intelligence Manager, Delphix Mike Donovan | ChiefTechnicalOfficer, DBVisit| How to Eliminate the Storm When Moving to the Cloud
- 2. © 2017 Delphix.All Rights Reserved. Private and Confidential. 2 Kellyn Pot’Vin-Gorman Technical Intelligence Manager, Delphix • Multi-platform DBA, (Oracle, MSSQL, MySQL, Sybase, Postgres…..) • Oracle ACE Director, (Alumni) • Oak Table Network • APEX Women in Technology Award, CTA 2014 • STEM education with Raspberry Pi and Python • President, Rocky Mtn Oracle User Group • Liaison, Denver SQL Server User Group • Author, blogger, (http://dbakevlar.com)
- 3. © 2017 Delphix.All Rights Reserved. Private and Confidential. -Per Evans Research sinceTracking in 2009
- 4. © 2017 Delphix.All Rights Reserved. Private and Confidential. Moving to the Cloud is Hard • 66% of companies are delaying migrations to the cloud. • 44% of cloud migrations are deemed failures. • 74% who did migrate over suffered outages of 48 hours or more during the transition. https://betanews.com/2016/08/05/professionals-delaying-migration/
- 5. © 2017 Delphix.All Rights Reserved. Private and Confidential. Agility with DevOps • 20%+ increase to the development cycle withAgile and DevOps. • Almost 25% increase in collaboration between departments. • Almost 50% increase in revenue with DevOps. • 60% of companies are utilizing 2-5 clouds on average. http://www.devx.com/enterprise/devops-drives-revenue-for-it-shops-ca-study.html
- 6. © 2017 Delphix.All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential. The promise of the cloud is huge, but the journey isn’t easy. Cloud value cannot be achieved through technology alone—it’s about governance, security, people, and processes. This report validates that support for cloud in federal agencies is growing, but perceptions of its impact vary significantly. Agencies need to rethink their core development principles and strategy for going cloud native. – Doug Bourgeois, managing director, Deloitte Consulting LLP
- 7. © 2017 Delphix.All Rights Reserved. Private and Confidential. Data Causes Friction • If those managing the data sources aren’t included and working well with those needing that data to produce features and products, friction is the result. • DBAs are consistently working with archaic utilities that guarantee outcome, rarely speed of results.
- 8. © 2017 Delphix.All Rights Reserved. Private and Confidential.
- 9. © 2017 Delphix.All Rights Reserved. Private and Confidential. Secondary Data Gravity? Data Gravity is the ability for data to attract applications, services, etc. As with the laws of physical gravity, data, due to its mass, will attract all else that has less mass.
- 10. © 2017 Delphix.All Rights Reserved. Private and Confidential. When you consider… An estimate of a 10% increase in data accessibility will result in more than $65 million in additional net income, you can see why data gravity is a big deal.
- 11. © 2017 Delphix.All Rights Reserved. Private and Confidential. How much data are we talking about? • More data has been created in just the last two years than the previous history of humanity • 1.7MB of new information created every second per human on the planet. • Introduction of big data often has same development pain points.
- 12. © 2017 Delphix.All Rights Reserved. Private and Confidential. Our Example Cloud StorageWe’re going to have to get everything to the cloud!
- 13. © 2017 Delphix.All Rights Reserved. Private and Confidential. A Solution to the Challenge
- 14. © 2017 Delphix.All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential. Virtualize
- 15. © 2017 Delphix.All Rights Reserved. Private and Confidential. Not Your Father’s Virtualization • NotVirtualized Servers, (VMs) • Not to be confused withVirtualizing a network • Not Federating Data • Not an ETL or subset of data
- 16. © 2017 Delphix.All Rights Reserved. Private and Confidential. Older Cloning Methods
- 17. © 2017 Delphix.All Rights Reserved. Private and Confidential. Who Has Time for THAT? • Copies requires adequate storage. • Requires more resources- demands on people and technology • Agile moves fast and data, (traditionally) moves slow. • More inline with archaic data movement tools.
- 18. © 2017 Delphix.All Rights Reserved. Private and Confidential. Virtualize Compress and Dedup
- 19. © 2017 Delphix.All Rights Reserved. Private and Confidential. Create as Many Copies as Needed • No more waiting for a physical clone • Uses the same native backup files that DBAs depend on, but in a new way • Free up the DBA to work on others tasks • Development, testing, QA, Reporting, Offload of Batch and ETL processing • Test backups and simplify patching and upgrading.
- 20. © 2017 Delphix.All Rights Reserved. Private and Confidential. Virtualized Database, (VDB)
- 21. © 2017 Delphix.All Rights Reserved. Private and Confidential. Now to Escape the Data Gravity • Provision in minutes, not days or weeks • Refreshes almost instantaneously • Self-service portal for developers and testers to refresh and develop they way they do in an agile environment. • Tools with an interface made for Agile and Development data at its focus.
- 22. © 2017 Delphix.All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential. Replicate
- 23. © 2017 Delphix.All Rights Reserved. Private and Confidential. Install DBVisit Replication on Source and Target Source DB server SQL Server 2008-2016 1 TB Delphix Virtualization Engine 2 TB storage Target DB server SQL Server 1008-2016 No database storage Bin files, flat files, data sources
- 24. © 2017 Delphix.All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential. Containerize
- 25. © 2017 Delphix.All Rights Reserved. Private and Confidential. Containers Come in Many Forms Containers != Docker Only Containerizing = Packaging Not Just for Hosts
- 26. © 2017 Delphix.All Rights Reserved. Private and Confidential. Group Environments into Data Pods
- 27. © 2017 Delphix.All Rights Reserved. Private and Confidential. Why Create Data Pods • Development is done by project or feature vs. tier or product. • With the introduction to the cloud, the business requires those that can do more at a global scale. • Ease of management and maintenance • Provide more value to the business
- 28. © 2017 Delphix.All Rights Reserved. Private and Confidential. Data Virtualization, OnPrem & Data Pods Source DB server SQL Server 2008-2016 1 TB Delphix Virtualization Engine 2 TB storage Target DB server SQL Server 1008-2016 No database storage SCCI/SSL Backups via SMB Bin files, flat files, data sources Virtualized and now containerized as a Data Pod
- 29. © 2017 Delphix.All Rights Reserved. Private and Confidential. Data Virtualization, OnPrem & Data Pods Source DB server SQL Server 2008-2016 1 TB Delphix Virtualization Engine 2 TB storage Target DB server SQL Server 1008-2016 No database storage SCCI/SSL Backups via SMB Bin files, flat files, data sources Virtualized and now containerized as a Data Pod
- 30. © 2017 Delphix.All Rights Reserved. Private and Confidential. Data virtualization: from on-premises into the Cloud Source DB server SQL Server 2008-2016 2TB storage Delphix Data Platform 2 TB storage Target DB server SQL Server 2008-2016 No database storage
- 31. © 2017 Delphix.All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential. Keep Up-to-Date
- 32. © 2017 Delphix.All Rights Reserved. Private and Confidential. DBVisit Replication keeps cloud data up to date as project progresses Source DB server SQL Server 2008-2016 2TB storage Delphix Data Platform 2 TB storage Target DB server SQL Server 2008-2016 No database storage
- 33. © 2017 Delphix.All Rights Reserved. Private and Confidential.© 2017 Delphix. All Rights Reserved. Private and Confidential. Solution- High Level
- 34. © 2017 Delphix.All Rights Reserved. Private and Confidential. The Power of Data Pods Personal Secure Portable DynamicLightweight
- 35. © 2017 Delphix.All Rights Reserved. Private and Confidential. Data Pods for Cloud Migrations O N P R E M D B A D E V Q A C L O U D
- 36. © 2017 Delphix.All Rights Reserved. Private and Confidential. Data Pods for Cloud Migration O N P R E M D E V Q A C L O U D D B A
- 37. © 2017 Delphix.All Rights Reserved. Private and Confidential. Data Pods for Cloud Migration O N P R E M D E V Q A C L O U D D B A
- 38. © 2017 Delphix.All Rights Reserved. Private and Confidential. Delphix Platform for Cloud Adoption O N P R E M D E V Q A C L O U D D B A
- 39. © 2017 Delphix.All Rights Reserved. Private and Confidential. Delphix Platform for Cloud Adoption O N P R E M D E V Q A C L O U D D B A
- 40. © 2017 Delphix.All Rights Reserved. Private and Confidential. Delphix Platform for Cloud Adoption O N P R E M Q A C L O U D D B A D E V
- 41. © 2017 Delphix.All Rights Reserved. Private and Confidential. The Data Pod Change • Development evolves from data gravity to DevOps focused with DataOps. • Enable the business by removing data gravity and the friction it causes by • Virtualizing environments • Creating Data Pods to simplify and empower environment management.
- 42. © 2017 Delphix.All Rights Reserved. Private and Confidential. A Working Solution • Solution incorporates applications, support files and replication, like DBVisit, not just database tier. • Allows for multi-faceted approach to cloud migrations • Limits storage cost, storage footprint and data in flight costs that is a consistent hindrance to cloud migration success.
- 43. © 2017 Delphix.All Rights Reserved. Private and Confidential. Use DBVisit thank you and closing slides…
Editor's Notes
- #3 Need Mike’s intro slide…
- #4 83% of development is currently performed in the cloud, so we can’t discuss development today without including the cloud. Delays are not an option- including those in hardware and IT allocation. This pushes a need for the cloud that previously we hadn’t experienced. The ability for developers to allocate an environment and start working within minutes is essential to agile
- #5 Of the 935 professionals surveyed, 35 percent say that they lack the experience or the expertise to confidently perform a system migration. No surprise then that a worrying 44 percent of businesses experienced a migration failure in 2015-2017 In addition, 57 percent of migrations required more resources to perform the migration.
- #6 I speak to many DBAs about DevOps as the third option for the future of their career. It’s not just about operations or development, but a hybrid of both and the power of automation. Our natural skill sets will benefit this. Now data is at the center of the hold on DBAs, but it can be mitigated and eliminated. Really? You’re all saying this, I know.
- #8 Data is center of this. I’m a DBA, I know this. Data sources are central to most development projects and as applications and code is migrated to the cloud, so must the data source. For a typical Fortune 1000 company, just a 10% increase in data accessibility will result in more than $65 million additional net income. Leveraging data coupld increase revenue by as much as 60%
- #9 Data Gravity has two definitions and both revolve around data sources. For the first, there are larger data sources every day. Databases are at the center of this friction and the natural life of a database is growth. By 2020, a third of all data will be on the cloud and 58% of data will be comprised in big data. In the last two years, we’ve created more data has been created in the past two years than in the entire previous history of the human race. Data isn’t going to slow down, either. By the year 2020, about 1.7 megabytes of new information will be created every second for every human being on the planet. By 2020, we’ll grow from today’s 4.4 zettabyets to an approximate, but staggering 44 zettabytes, or 44 trillion gigabytes. And by 2020, a third of that data will pass through the cloud.
- #10 For the second, Data gravity is the ability of bodies of data to attract applications, services and other data. ... IT expert Dave McRory coined the term data gravity as an analogy to the way that, in accordance with the physical laws of gravity, objects with more mass attract those with less.
- #11 And yet we state that we won’t need DBAs? That data isn’t the center of challenge?
- #12 Per Forbes, by the year 2020, about 1.7 megabytes of new information will be created every second for every human being on the planet. more data has been created in the past two years than in the entire previous history of the human race. That data has to be stored somewhere and there’s a large chance it’s going to be in a relational data store.
- #13 Everything on prem needs to move to the cloud. There’s commonly one over all project with three sub projects that will accomplish all of this. DBAs will be working to move the database and all the data. Application and development will work to migrate the application and support files Administrators will work to prepare the cloud environments if it’s IaaS.
- #14 Data gravity suffers from the Von Newmann Bottleneck. It’s a basic limitation on how fast computers can be. Pretty simple, but states that the speed of where data resides and where it’s worked with, (i.e. processed) is the limiting factor in computing speed. This is why we have numerous local cache to address memory issues while processing data. Microsoft researcher Jim Gray has spend most of his career looking at the economics of data, which I think is pretty cool! This is why DBAs spend so much time on the cost based optimizer, as moving or getting data has cost. After all is said and done, this is the fundamental principle of data gravity and why DBAs get the big bucks.
- #15 We’ll start with three areas that solve the problem of data gravity and data friction in agile development, We Virtualize.
- #17 RMAN duplicates, cold backup to restores, datapump and other archaic data transfer processes are time consuming.
- #19 By virtualizing, we remove the “weight” of the data. We know that 80% of the data won’t change between copies, so why do we need individual copies of it. Our source is then deduped and compressed to conserve more space.
- #21 Each Virtual Database, (VDB) will no longer require space, (only background and foreground memory for SGA/PGA, etc.) and local redo logs. This is a considerable savings, but… If we take this a step further by embracing write changes only on blocks changed from the source, then we’ll experience 10-20 copies of a database in about the same space that one database requires.
- #22 The business is able to provision new environments or refresh existing ones in a matter of minutes. Developers and testers who’ve worked with bookmarks and branching of their code changes can now do the same with database changes, rewinding and refreshing as they need without impacting the DBAs day. This allows the DBA to do more with their time. Having tools that includes the database in the Agile development cycle makes a pivotal change in how the DBA is capable of being part of DevOps.
- #24 ** Please replace DBVisit logo with a better graphic to describe the replication engine! Feel free to add more slides to go into full detail of what it does, etc. The goal is to say that we’ll add this replication to both the Dsource, (golden copy) and the target or virtual images/data pods.
- #27 The next step is moving to data pods. Containers are a buzz area of technology right now. If we’re talking Docker or Kubernetes, we know this is the way of the future. Instead of having locked, unique environments, the ability to package them as one, in a lighter and more flexible unit makes incredible sense. As a DBA, I rarely, if ever, just released code to the database. It was commonly to the database, the application and linked products. The ability to package and manage as a Data Pod is an impressive enhancement to the Developer, tester and DBA.
- #28 The next step is the ability to migrate to the cloud or from one cloud to another. Right now, 60% of customers are using 2-5 clouds on average. The ability to move a Data Pod from one cloud to another is incredibly powerful. Companies are spending increased time now just migrating to the cloud, but to other clouds and if it would be as simple as migrating a Data pod with a few changes to the new storage location, (i.e. cloud) that could save companies millions of dollars.
- #29 With virtualization, we virtualize it all- the database, data sources, application and flat files. We containerize it into a Data Pod
- #30 With virtualization, we virtualize it all- the database, data sources, application and flat files. We containerize it into a Data Pod
- #31 By doing so, it’s easier to life and shift to the cloud. It’s lighter and we can move 20 + environments in the same space as one.
- #33 Full refreshes are excellent, but for progressive deployments and releases, replication can offer an added layer to cloud migrations.
- #35 Need to fill this out with DBVisit diagrams and solutions into the dynamic data platform slides I’ve built…
- #42 So we evolve from being a DBA to embracing more DevOps skills, including more scripting, scheduling and automation, then include DataOps to create a flexible, agile solution. There are two areas that the DBA can build out on their path to DevOps that are database centric and crucial to the solution- Virtualize Data pods to package environments. By doing all these things, impressive changes can be made by the DBA to any team and they can become valued as a key member of the agile development cycle.