Message231745
To see what happens at syscall level, I've run various implementations of get_tree_size() functions (see get_tree_size_listdir.diff) with strace: get_tree_size_listdir_fd -- os.listdir(fd) + os.lstat get_tree_size -- os.scandir(path) + entry.lstat get_tree_size_listdir_stat -- os.listdir(path) + os.lstat get_tree_size_listdir -- os.listdir(path) + os.path.isdir + os.lstat Summary: - os.listdir() and os.scandir()-based variants use the same number of getdents() syscalls - and the number of openat, lstat (newfstatat) syscalls is also comparable (except for the variant that uses os.path.isdir) Log: scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir_fd as f; import os; print(f(os.open("/usr/", os.O_RDONLY)))' && head -7 py.strace 5535240217 11.29user 8.14system 0:17.78elapsed 109%CPU (0avgtext+0avgdata 13460maxresident)k 0inputs+8outputs (0major+6781minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 46.51 0.075252 0 264839 newfstatat 16.88 0.027315 1 50460 getdents 11.53 0.018660 0 75621 fcntl 9.74 0.015758 0 50531 close 6.87 0.011116 0 25214 openat scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size as f; print(f("/usr/"))' && head -7 py.strace 5535240217 22.56user 8.47system 0:29.77elapsed 104%CPU (0avgtext+0avgdata 13280maxresident)k 0inputs+8outputs (0major+6306minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 62.00 0.032822 0 239644 lstat 19.97 0.010570 0 50460 getdents 8.52 0.004510 0 25215 openat 6.09 0.003224 0 25326 close 0.55 0.000292 3 95 mmap scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir_stat as f; print(f("/usr/"))' && head -7 py.strace 5535240217 13.70user 6.30system 0:18.84elapsed 106%CPU (0avgtext+0avgdata 13456maxresident)k 0inputs+8outputs (0major+6769minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 64.79 0.050217 0 264849 lstat 19.37 0.015011 0 50460 getdents 8.22 0.006367 0 25215 openat 5.76 0.004465 0 25326 close 0.32 0.000247 2 114 mmap scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir as f; print(f("/usr/"))' && head -7 py.strace 5535240217 19.53user 10.61system 0:28.16elapsed 107%CPU (0avgtext+0avgdata 13452maxresident)k 0inputs+8outputs (0major+6733minor)pagefaults 0swaps % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 42.04 0.063050 0 265195 37259 stat 37.40 0.056086 0 265381 lstat 11.46 0.017187 0 50460 getdents 4.82 0.007232 0 25215 openat 3.43 0.005139 0 25326 close | |
| Date | User | Action | Args | | 2014-11-27 04:31:13 | akira | set | recipients: + akira, tebeka, pitrou, vstinner, giampaolo.rodola, tim.golden, benhoyt, abacabadabacaba, socketpair, josh.r | | 2014-11-27 04:31:13 | akira | set | messageid: <1417062673.14.0.908320154024.issue22524@psf.upfronthosting.co.za> | | 2014-11-27 04:31:13 | akira | link | issue22524 messages | | 2014-11-27 04:31:12 | akira | create | | |