Download as PDF, PPTX
Uploaded byAlex Chistyakov
Python performance engineering in 2017
This document discusses profiling Python performance. It begins by acknowledging that Python can be slow due to the Global Interpreter Lock (GIL) and lack of a just-in-time (JIT) compiler. It then demonstrates how to profile a Python program using the pyflame tool to collect stack samples and convert them to a flamegraph for analysis. The document shows pyflame being used to profile a simple Sanic web application under load testing. It finds that pyflame has surprisingly little overhead and concludes that while Python has limitations, it is not inherently slow when profiled and optimized properly.
Python performance engineering in 2017
- 1. Python performance engineering in2017 Alex Chistyakov, an engineer @ some company
- 2. Agenda - Is Pythonslow? - How to profile a Python program? - We are not going to talk about blockchain, sorry 2
- 3. Is Python slow? -Definitely, because of GIL and lack of proper JIT 3
- 4. Is Python slow? -Definitely, because of GIL and a lack of a proper JIT compiler 4
- 5. Is Python slow? -Seriously, how do you define “to be slow”? 5
- 6. Okay, let’s doour homework finally - Vagrant + https://github.com/alexclear/piterpy2017 6
- 7. Okay, let’s doour homework finally - Vagrant + https://github.com/alexclear/piterpy2017 - 2-cores VirtualBox VM 7
- 8. Okay, let’s doour homework finally - Vagrant + https://github.com/alexclear/piterpy2017 - 2-cores VirtualBox VM - Something called Sanic: https://github.com/channelcat/sanic because it’s 2017 and 2.x era should have ended years ago 8
- 9. Okay, let’s doour homework finally - Vagrant + https://github.com/alexclear/piterpy2017 - 2-cores VirtualBox VM - Something called Sanic: https://github.com/channelcat/sanic because it’s 2017 and 2.x era should have ended years ago - A load testing tool called ab because we love classics 9
- 10. What is atypical Python program? - https://goo.gl/Abz8LX (Please note that this link will produce different results in 5 years) - Something not too server-side (or a framework) - Something scientifical - Some utility 10
- 11. What is atypical Python program? - https://goo.gl/Abz8LX (Please note that this link will produce different results in 5 years) - Something not too server-side - Something scientifical - Some utility - An async web application! 11
- 12. A crash coursein modern profiling - Collect stack samples at all costs 12
- 13. A crash coursein modern profiling - Collect stack samples at all costs - Convert collected samples to Brendan Gregg’s flamegraph.pl format 13
- 14. A crash coursein modern profiling - Collect stack samples at all costs - Convert collected samples to Brendan Gregg’s flamegraph.pl format - Create a flamegraph 14
- 15.
- 16. Collecting samples - Uber’spyflame: https://github.com/uber/pyflame 16
- 17. A bit ofinternal dark magic - pyflame uses ptrace(2) system call 17
- 18. A bit ofinternal dark magic - pyflame uses ptrace(2) system call - It attaches to a running process and traverses its memory 18
- 19. Limitations - Linux only -CPython only - Should have enough privileges - Can be slow (again, what is “to be slow”? I honestly have no idea) 19
- 20. Enough on that,let’s get real - Starting the server: python3 ./sanic-app.py 1>/dev/null 2>/dev/null - (It does basically nothing, just “hello world” stuff) 20
- 21. Enough on that,let’s get real - Stressing the server: ab -n 300000 -c 300 http://127.0.0.1:8000/ - Please note that using ab for real world simulations is totally insane, but this is exactly what we want now 21
- 22. How slow ispyflame? - We run tests with and without pyflame 22
- 23.
- 24. Conclusions - Pyflame issurprisingly fast! - Python is not slow, it’s just drawn that way 24
- 25. Questions, please? - alexclear@gmail.com -Skype, Telegram: demeliorator - Shameless advertising: http://t.me/lhommequipleure 25