Skip to content

Commit 6039919

Browse files
authored
Adjust paramiko and bcrypt versions (#115)
Currently the OS default package manager installs paramiko version 2.12.0 with the following instruction when UPGRADE_PARAMIKO is set to False https://github.com/ucphhpc/docker-migrid/blob/4cb2c075daad5cd2a4eefe864a978b86a3b68029/Dockerfile.rocky9#L419 However, this has shown to produce up to 6 times less SFTP based download bandwidth compared to uploading. As indicated at https://www.paramiko.org/changelog.html, version 3.0.0 makes several improvements to the SFTP performance, that fixes this imbalance in our benchmarks. To implement this version fix, it would be as simple as rebuilding the image the UPGRADE_PARAMIKO=True that ensure that the latest paramiko version available at pypi.org is installed via: https://github.com/ucphhpc/docker-migrid/blob/4cb2c075daad5cd2a4eefe864a978b86a3b68029/Dockerfile.rocky9#L771 This however has shown the side-effect of producing the https://github.com/PyO3/pyo3/blob/main/guide/src/migration.md#each-pymodule-can-now-only-be-initialized-once-per-process import errors by the mod_wsgi process from time to time when importing the paramiko module. This is likely due to mod_wsgi usage of sub-interpreters and the failure to reinitialize the import in a different interpreter after the first initialization. For the time being, the fix is to limit the bcrypt version to be less than 4.x which Rust implementation seems to introduce this error.
2 parents 4cb2c07 + 7c68466 commit 6039919

File tree

1 file changed

+14
-1
lines changed

1 file changed

+14
-1
lines changed

Dockerfile.rocky9

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -768,7 +768,20 @@ RUN if [ "${UPGRADE_PARAMIKO}" = "True" ]; then \
768768
# dependencies specifically into /usr instead.
769769
#pip3 install -U setuptools; \
770770
#pip3 install setuptools_rust; \
771-
pip3 install --prefix=$(python3-config --prefix) paramiko; \
771+
# Additional NOTE: paramiko version 3.x introduces several changes
772+
# that in particular improve the SFTP download performance.
773+
# At the time of writing, the RHEL/Rocky 10 EPEL repo provides 3.5,
774+
# which we use as a version baseline to mimic.
775+
# However, combining that with recent bcrypt versions in the 4.x
776+
# series leads to import errors in mod_wsgi caused by the usage of
777+
# sub-interpreters as highlighted at
778+
# https://github.com/PyO3/pyo3/blob/main/guide/src/migration.md#each-pymodule-can-now-only-be-initialized-once-per-process
779+
# with further explanation at
780+
# https://bugzilla.redhat.com/show_bug.cgi?id=2255688 and
781+
# https://github.com/PyO3/pyo3/issues/3451
782+
# So we explicitly cap the bcrypt version to the latest 3.2 one,
783+
# which coincides with the default Rocky version, btw.
784+
pip3 install --prefix=$(python3-config --prefix) 'paramiko>=3.5' 'bcrypt<4'; \
772785
fi; \
773786
fi;
774787

0 commit comments

Comments
 (0)