Downloaded 13 times
Uploaded byGreat Wide Open
Running MySQL on Linux
The document discusses best practices for running MySQL on Linux, covering choices for Linux distributions, hardware recommendations including using solid state drives, OS configuration such as tuning the filesystem and IO scheduler, and MySQL installation and configuration options. It provides guidance on topics like virtualization, networking, and MySQL variants to help ensure successful and high performance deployment of MySQL on Linux.
Running MySQL on Linux
- 1. Running MySQL on Linux Peter Zaitsev CEO, Percona Great Wide Open Atlanta,GA 16 March 2016
- 2. About Percona Solu@ons for Success with MySQL and MongoDB Support, Consul@ng, Managed Services SoIware for Success with Data Management 100% Open Source, no lock-‐in required
- 3. About the Presenta8on • Distribu@on • Hardware • OS Configura@on • MySQL Installa@on • MySQL Configura@on Cover what you need to run MySQL on Linux successfully 3
- 4. Why Linux ? Most Popular PlaVorm for Produc@on Deployments The main plaVorm for MySQL Engineering Efforts Great Default Choice! 4
- 5. Choices • Long Support; Stability “Server Grade Distribu@on” • Especially with Newer Hardware Recent enough • RHEL/CentOS/Oracle/Amazon • Debian, Ubuntu Most Popular Choices
- 6. Things to Consider Low Overhead monitoring support “perf”, Irace, eBPF NVMe Docker Support File System Encryp@on New Block Device Infrastructure SoIware RAID 6
- 7. Hardware • Really! Hardware can go a long way! • With 100 queries/page = 120K page views/minute • Some 80M page views/day (considering daily spike) Over 200K simple queries/ sec on modern hardware 7
- 8. MySQL 5.7 PerformanceImprovements Sysbench Benchmark *Informa)on from Oracle OpenWorld presenta)on by Geir Hoydalsvik Starts with 8 Threads What about 2-‐4 threads?
- 9. Hardware • Go for fast cores. All cores are rarely used these days • Cache and fast memory bus are important CPU • OIen most important for performance • Your working set must fit in memory well • Less memory = more pressure on IO Memory 9
- 10. Choose Solid State for Main Storage Proprietary PCE-‐E and NVMe SATA SSDs very inexpensive SoIware RAID to beat SATA limits SSDs available in the cloud – use them Not all Flash is created equal 10
- 11. More on SSD/Flash Think about Write Endurance Behavior on Power Loss Data Reten@on Firmware 11
- 12. Know your Goals • Reads? • Writes? • Random? • Sequen@al? Are you bound by • Sustained writes? • Handle spikes? Do you need • Many devices reach peak IOPs at unreachable concurrency What concurrency do you operate? • SAN/NAS is a frequent cause of surprise Think about latency/ response @me 12
- 13.
- 14. Compression of Performance Gains 14
- 15. Network Latency is king Minimize number of hops between Database and Web Server Ensure running 1Gbit link speed at least (10Gb is gaining popularity) Monitor for packet loss and latency Network problems are oIen blamed on database 15
- 16. Network Tuning Might be needed some@mes • echo 8192 > /proc/sys/net/ipv4/tcp_max_syn_backlog • Mysql: back_log=1000 • net.ipv4.ip_local_port_range="1024 64000“ • net.core.somaxconn = 1024 • net.core.rmem_max = 16777216 • net.core.wmem_max = 16777216 • net.ipv4.tcp_rmem = 4096 87380 16777216 • net.ipv4.tcp_wmem = 4096 65536 16777216 • net.ipv4.max_tw_buckets=360000 • net.core.netdev_max_backlog = 2500 • ifconfig eth0 txqueuelen 1000 Persistent connec@ons with MySQL • Thread_pool (Percona Server, MariaDB, MySQL Enterprise) • Proxy 16
- 17. Virtualiza8on and Cloud Virtualiza@on has cost Cloud rarely provides highest performance hardware (Getng beuer) There is a lot of MySQL use in the Cloud and Virtualized Environments 17
- 18.
- 19. In General… For most workloads Linux Runs MySQL surprisingly well with no addi@onal tuning 19
- 20. General Configura8on • vm.swappiness=0 Reduce Tendency to swap 20
- 21. General Configura8on Reduce the File Cache • vm.dirty_ra@o=5 NUMA can cause problems • hup://bit.ly/I0GSt3 • Interleaving with numactl • MySQL 5.6+ Supports innodb_numa_interleave 21
- 22. Storage Configura8on • Re-‐Configure MySQL without OS Reinstall Separate OS and MySQL Par@@on • CFQ can cause problems • Deadline or Noop are oIen beuer choice • elevator=deadline on kernel boot Configure IO Scheduler • # echo 100000 > /sys/block/sdX/queue/nr_requests Queue length (especially MyISAM) • Virtually no overhead when not in snapshot mode • Reserve some space for snapshots Consider using LVM 22
- 23.
- 24. Filesystems XFS – “Old” High Performance Favorite EXT4 – Works very good for some workloads Avoid: • Ext2 • Ext3 • ReiserFS • ZFS (may be good for development) • Btrfs 24
- 25. FS Tuning rela@me (noa@me) ext3: tune2fs –O dir_index -‐c –l –i 0 –e remount-‐ro xfs: nobarrier • Assuming RAID w BBU or Flash 25
- 26. WHAT MYSQL VARIANT ARE YOU RUNING ? 26
- 27. Main Choices MySQL Percona Server MariaDB 27
- 28. MySQL Versions Newer versions scale beuer MySQL 5.7 is most scalable Percona Server 5.7 brings even more improvements J Use 5.7 for New Development Upgrade to MySQL 5.7 if you need it now 28
- 29. Check out Percona XtraDB Cluster Based on Galera Technology Innodb Storage Engine you already know Synchronous Replica@on Parallel Apply Automa@c Provisioning No Stale Reads Not for all workloads 29
- 30. How to Install • hup://dev.mysql.com/downloads/repo/ • hup://www.percona.com/doc/percona-‐server/5.7/installa@on.html Use Repositories: Best • Need older Version Download RPM/DEB • Good for tes@ng with MySQL Sandbox Use Tar.gz package • hups://hub.docker.com/_/percona/ Docker Images • Do you really have good reason to do it ? • Many problems are caused by bad builds Build our own 30
- 31. MySQL Configura8on • Even newer MySQL 5.7 defaults are unlikely to be op@mal You need to tune MySQL configura@on • hup://bit.ly/1fuP0SZ Check out presenta@on which goes in depth into configura@on tuning • Avoid obsessive tuning disorder Typically getng 5 variables right responsible for 90% performance gain 31
- 32. MySQL Variables • How many connec@ons server will support ? max_connec@ons • How many “open” tables there will be allowed table_open_cache • How many files can be open open_files_limit 32
- 33. Top Variables • Set 80% of memory some@mes more Innodb_buffer_pool_size • Best setng in most cases Innodb_flush_method=O_DIRECT • Set 256MB or more • Larger logs = longer recovery @me Innodb_log_file_size • Truly ACID: 1 • Can Afford Data Loss: 2 Innodb_flush_log_at_trx_commit=? 33
- 34. Advanced Ideas • taskset -‐pc 0,12,2,14 `cat /var/ lib/mysql/mysqld1.pid` Running mul@ple MySQL instances ? • Jemalloc • Tcmalloc –old favorite Memory Alloca@on Hostspot ? • Roll back easily if things go wrong LVM Snapshot before maintenance or upgrade 34
- 35. Beware of the “Scripts” • Timeouts might be not enough for safe database shutdown • The automated upgrade/check Start/ Stop Scripts can be nasty 35
- 36. Automa8on Manual approach is a no • Does not Scale • And is Error Prone Automate installa@on, upgrades, configura@on • Puppet, • Chef • Ansible Keep Configura@on under version control • At very least leave comments on what you chance and why 36
- 37. Monitoring and Trending • Just make sure you’re doing it Many tools to choose from! • Check out PMP • hup://www.percona.com/soIware/percona-‐monitoring-‐ plugins Nagios and CACTI • hup://grafana.org/ Grafana – new genera@on trending • hup://bit.ly/1M6r7Vk Check out Grafana + Prometheus Monitoring 37
- 38. Prometheus + Grafana Example 38
- 39. Linux OOM Killer • SSH • MySQL Server • Various jobs • Backup • Percona Toolkit • Background Batch jobs What are your priori@es ? • echo -‐17 > /proc/2592/oom_adj • do not kill ever • echo 10 > /proc/2592/oom_adj • More like to be killed Configuring 39
- 40. Tools you need to know General Linux • Top • Vmstat • Iostat • Strace • Gdb • perf MySQL • Percona Toolkit • Innotop • Percona Xtrabackup • MyDumper 40
- 41. Resources for MySQL on Linux • Brendan Gregg’s web site – hup://www.brendangregg.com/ • Percona’s Blog – hup://www.percona.com/blog/ • Ernie Souhrada IOPS inves@ga@on – bit.ly/2546xes 41
- 42. www.percona.com Percona Live Data Performance Conference • April 18-‐21 in Santa Clara, CA at the Santa Clara Conven@on Center • Register with code “GWO” to receive 15% off at registra@on • MySQL, NoSQL, Data in the Cloud www.perconalive.com
- 43. 43 Thank You! Peter Zaitsev pz@percona.com @PeterZaitsev bit.ly/PerconaJobs