MissingHeaderBodySeparatorDefect when uploading items to OSS #622
Description
When uploading files to OSS, I randomly (~1000 items in) experience this error after I run
with open(file_path, 'rb') as f: obj_data = f.read() try: response = oci_client.put_object( OSS_NAMESPACE, OSS_BUCKET_NAME, oss_object_name, # object name obj_data, # object body (bytes) ) if response.status != 200: msg = f'WARNING: {oss_object_name} upload resulted in status {response.status}. {response.data}' logging.log(logging.WARN, msg) print(msg) else: msg = f'SUCCESS! {oss_object_name} uploaded' logging.log(logging.INFO, msg) print(msg) except oci.exceptions.ServiceError as e: # Handling OCI specific exceptions msg = f'ERROR: {oss_object_name} upload failed with ServiceError. {e.status} {e.message}' logging.error(msg) print(msg) except Exception as e: msg = f'ERROR: {oss_object_name} upload failed. {e}' logging.log(logging.ERROR, msg) print(msg) Here's the traceback:
2024-02-19 20:09:39,991:WARNING:Failed to parse headers (url=https://objectstorage.ap-osaka-1.oraclecloud.com:443/n/ax2tjlcefnht/b/laion2B-en-aesthetic/o/000000401/000000401.txt): [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 200 OK\r\netag: 07f8f92b-fd96-49ec-b9db-1b927e144116\r\nlast-modified: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-content-md5: tMGI50PK84P9//D2pQ8evQ==\r\nversion-id: 04303300-a3ec-4fdb-a2eb-58cd65effb06\r\nContent-Length: 0\r\ndate: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-request-id: kix-1:BaX2PWrWLWStJYPKMWmC_TRLmcJaINqwJQey95jbJrG80yzz4rVgEBGGUeet-gvk\r\nx-api-id: native\r\nx-content-type-options: nosniff\r\nstrict-transport-security: max-age=31536000; includeSubDomains\r\naccess-control-allow-origin: *\r\naccess-control-allow-methods: POST,PUT,GET,HEAD,DELETE,OPTIONS\r\naccess-control-allow-credentials: true\r\naccess-control-expose-headers: access-control-allow-credentials,access-control-allow-methods,access-control-allow-origin,content-length,date,etag,last-modified,opc-client-info,opc-content-md5,opc-request-id,strict-transport-security,version-id,x-api-id,x-content-type-options\r\n\r\n' Traceback (most recent call last): File "/home/Mindy/.cache/pypoetry/virtualenvs/t2i-adapter-3QVI5i9t-py3.10/lib/python3.10/site-packages/oci/_vendor/urllib3/connectionpool.py", line 474, in _make_request assert_header_parsing(httplib_response.msg) File "/home/Mindy/.cache/pypoetry/virtualenvs/t2i-adapter-3QVI5i9t-py3.10/lib/python3.10/site-packages/oci/_vendor/urllib3/util/response.py", line 96, in assert_header_parsing raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data) oci._vendor.urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 200 OK\r\netag: 07f8f92b-fd96-49ec-b9db-1b927e144116\r\nlast-modified: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-content-md5: tMGI50PK84P9//D2pQ8evQ==\r\nversion-id: 04303300-a3ec-4fdb-a2eb-58cd65effb06\r\nContent-Length: 0\r\ndate: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-request-id: kix-1:BaX2PWrWLWStJYPKMWmC_TRLmcJaINqwJQey95jbJrG80yzz4rVgEBGGUeet-gvk\r\nx-api-id: native\r\nx-content-type-options: nosniff\r\nstrict-transport-security: max-age=31536000; includeSubDomains\r\naccess-control-allow-origin: *\r\naccess-control-allow-methods: POST,PUT,GET,HEAD,DELETE,OPTIONS\r\naccess-control-allow-credentials: true\r\naccess-control-expose-headers: access-control-allow-credentials,access-control-allow-methods,access-control-allow-origin,content-length,date,etag,last-modified,opc-client-info,opc-content-md5,opc-request-id,strict-transport-security,version-id,x-api-id,x-content-type-options\r\n\r\n' Even after adding try...except, I can't catch the exception and my code hangs. put_object worked fine for me two weeks ago when I uploaded hundreds of thousands of items.