The libioil library can be used to improve the performance of read and write operations to DFuse from applications which use libc. The library bypasses the kernel by intercepting POSIX read and write calls from the application and servicing them directly in user-space.
Using libioil
To use the interception library, set LD_PRELOAD to point to the shared library in the DAOS install directory:
LD_PRELOAD=/usr/lib64/libioil.so For example:
$ dd if=/dev/zero of=./foo bs=1G count=20 20+0 records in 20+0 records out 21474836480 bytes (21 GB, 20 GiB) copied, 45.6925 s, 470 MB/s $ LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=20 20+0 records in 20+0 records out 21474836480 bytes (21 GB, 20 GiB) copied, 15.735 s, 1.4 GB/s It is also possible to link applications with the interception library at compile time with the -lioil flag.
For evaluating performance using the interception library, see Performance considerations and Test performance.
Monitor activity
To enable activity reports for the interception library, set the D_IL_REPORT environment variable. This returns a short summary of activity in the shared library destructor, typically as a program exits.
The value assigned to D_IL_REPORT defines the number of read and write calls to log to stderr. To log all calls, set the value to -1. To print the summary at program exit only, set the value to 0.
For example:
$ D_IL_REPORT=1 LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=5 [libioil] Intercepting write of size 1073741824 5+0 records in 5+0 records out 5368709120 bytes (5.4 GB, 5.0 GiB) copied, 4.13708 s, 1.3 GB/s $ D_IL_REPORT=3 LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=5 [libioil] Intercepting write of size 1073741824 [libioil] Intercepting write of size 1073741824 [libioil] Intercepting write of size 1073741824 5+0 records in 5+0 records out 5368709120 bytes (5.4 GB, 5.0 GiB) copied, 4.18626 s, 1.3 GB/s $ D_IL_REPORT=-1 LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=5 [libioil] Intercepting write of size 1073741824 [libioil] Intercepting write of size 1073741824 [libioil] Intercepting write of size 1073741824 [libioil] Intercepting write of size 1073741824 [libioil] Intercepting write of size 1073741824 5+0 records in 5+0 records out 5368709120 bytes (5.4 GB, 5.0 GiB) copied, 4.12114 s, 1.3 GB/s Some programs, including most GNU utilities from the coreutils package, have a destructor function to close stderr on exit. This means that, while the interception library works for many basic commands such as cp and cat, it's not possible to see the summary generated by the interception library.