This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author akira
Recipients abacabadabacaba, akira, benhoyt, giampaolo.rodola, josh.r, pitrou, socketpair, tebeka, tim.golden, vstinner
Date 2014-11-27.04:31:11
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1417062673.14.0.908320154024.issue22524@psf.upfronthosting.co.za>
In-reply-to
Content
To see what happens at syscall level, I've run various implementations of get_tree_size() functions (see get_tree_size_listdir.diff) with strace: get_tree_size_listdir_fd -- os.listdir(fd) + os.lstat get_tree_size -- os.scandir(path) + entry.lstat get_tree_size_listdir_stat -- os.listdir(path) + os.lstat get_tree_size_listdir -- os.listdir(path) + os.path.isdir + os.lstat Summary: - os.listdir() and os.scandir()-based variants use the same number of getdents() syscalls - and the number of openat, lstat (newfstatat) syscalls is also comparable (except for the variant that uses os.path.isdir) Log: scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir_fd as f; import os; print(f(os.open("/usr/", os.O_RDONLY)))' && head -7 py.strace 5535240217 11.29user 8.14system 0:17.78elapsed 109%CPU (0avgtext+0avgdata 13460maxresident)k 0inputs+8outputs (0major+6781minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 46.51 0.075252 0 264839 newfstatat 16.88 0.027315 1 50460 getdents 11.53 0.018660 0 75621 fcntl 9.74 0.015758 0 50531 close 6.87 0.011116 0 25214 openat scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size as f; print(f("/usr/"))' && head -7 py.strace 5535240217 22.56user 8.47system 0:29.77elapsed 104%CPU (0avgtext+0avgdata 13280maxresident)k 0inputs+8outputs (0major+6306minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 62.00 0.032822 0 239644 lstat 19.97 0.010570 0 50460 getdents 8.52 0.004510 0 25215 openat 6.09 0.003224 0 25326 close 0.55 0.000292 3 95 mmap scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir_stat as f; print(f("/usr/"))' && head -7 py.strace 5535240217 13.70user 6.30system 0:18.84elapsed 106%CPU (0avgtext+0avgdata 13456maxresident)k 0inputs+8outputs (0major+6769minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 64.79 0.050217 0 264849 lstat 19.37 0.015011 0 50460 getdents 8.22 0.006367 0 25215 openat 5.76 0.004465 0 25326 close 0.32 0.000247 2 114 mmap scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir as f; print(f("/usr/"))' && head -7 py.strace 5535240217 19.53user 10.61system 0:28.16elapsed 107%CPU (0avgtext+0avgdata 13452maxresident)k 0inputs+8outputs (0major+6733minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 42.04 0.063050 0 265195 37259 stat 37.40 0.056086 0 265381 lstat 11.46 0.017187 0 50460 getdents 4.82 0.007232 0 25215 openat 3.43 0.005139 0 25326 close
History
Date User Action Args
2014-11-27 04:31:13akirasetrecipients: + akira, tebeka, pitrou, vstinner, giampaolo.rodola, tim.golden, benhoyt, abacabadabacaba, socketpair, josh.r
2014-11-27 04:31:13akirasetmessageid: <1417062673.14.0.908320154024.issue22524@psf.upfronthosting.co.za>
2014-11-27 04:31:13akiralinkissue22524 messages
2014-11-27 04:31:12akiracreate