[20240219-0829] Upload not working for RDS-Web via REST-API
The user tries to upload files with each 3 GB. By using the browser the upload fails between 60-80 % and via REST API the upload also fails after 2.29 GB. Looking at the outputs of the commando line it seems like it is an SSL error.
First try:
(hsu) jan:~/coding/repositories/group-repo/code/cohrs/scripts python coscine_upload.py
_
(_)
___ ___ ___ ___ _ _ __ ___
/ __/ _ / _|/ | | ' \ / _
| (| () _ \ (| | | | | /
____/|/___|_|_| |_|___|
Coscine Python SDK 0.10.3 https://coscine.de/
S2B_MSIL1C_20180404T114639.nc: 71%|##########################################3 (closed) | 2.29G/3.25G [00:38<00:31, 30.1MB/s]urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:2427)
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/urllib3/connectionpool.py", line 875, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/urllib3/connectionpool.py", line 875, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/urllib3/connectionpool.py", line 875, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/urllib3/connectionpool.py", line 845, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='coscine.rwth-aachen.de', port=443): Max retries exceeded with url: /coscine/api/v2/projects/70b5855f-f81c-4956-8473-0f4a6e9275b3/resources/dbd90dba-5916-4b14-a39c-ed0a0a66eb2d/blobs/S2B_MSIL1C_20180404T114639.nc (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/client.py", line 414, in send_request response = self.session.send( ^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/requests_cache/session.py", line 229, in send response = self._send_and_cache(request, actions, cached_response, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/requests_cache/session.py", line 253, in _send_and_cache response = super().send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/requests/adapters.py", line 517, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='coscine.rwth-aachen.de', port=443): Max retries exceeded with url: /coscine/api/v2/projects/70b5855f-f81c-4956-8473-0f4a6e9275b3/resources/dbd90dba-5916-4b14-a39c-ed0a0a66eb2d/blobs/S2B_MSIL1C_20180404T114639.nc (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)')))
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/Users/jan/coding/repositories/group-repo/code/cohrs/scripts/coscine_upload.py", line 15, in resource.upload("S2B_MSIL1C_20180404T114639.nc", fp, form) File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/resource.py", line 613, in upload self._upload_blob(path, handle, progress) File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/resource.py", line 695, in _upload_blob self.client.post(uri, data=monitor, headers=headers) File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/client.py", line 554, in post return self.request("POST", *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/client.py", line 463, in request return self.send_request(request, stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/client.py", line 426, in send_request self.handle_request_exception(error) File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/client.py", line 479, in handle_request_exception raise ConnectionError( ConnectionError: Failed to connect to Coscine! Check your internet connection or whether Coscine is currently down. S2B_MSIL1C_20180404T114639.nc: 71%|##########################################3 (closed) | 2.29G/3.25G [00:38<00:16, 59.3MB/s]
Second try:
(hsu) jan:~/coding/repositories/group-repo/code/cohrs/scripts python coscine_upload.py
_
(_)
___ ___ ___ ___ _ _ __ ___
/ __/ _ / _|/ | | ' \ / _
| (| () _ \ (| | | | | /
____/|/___|_|_| |_|___|
Coscine Python SDK 0.10.3 https://coscine.de/
S2B_MSIL1C_20210620T133729.nc: 65%|####################################### | 2.28G/3.51G [00:37<01:06, 18.3MB/s]urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:2427)
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/urllib3/connectionpool.py", line 875, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/urllib3/connectionpool.py", line 875, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/urllib3/connectionpool.py", line 875, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/urllib3/connectionpool.py", line 845, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='coscine.rwth-aachen.de', port=443): Max retries exceeded with url: /coscine/api/v2/projects/70b5855f-f81c-4956-8473-0f4a6e9275b3/resources/dbd90dba-5916-4b14-a39c-ed0a0a66eb2d/blobs/S2B_MSIL1C_20210620T133729.nc (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/client.py", line 414, in send_request response = self.session.send( ^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/requests_cache/session.py", line 229, in send response = self._send_and_cache(request, actions, cached_response, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/requests_cache/session.py", line 253, in _send_and_cache response = super().send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/requests/adapters.py", line 517, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='coscine.rwth-aachen.de', port=443): Max retries exceeded with url: /coscine/api/v2/projects/70b5855f-f81c-4956-8473-0f4a6e9275b3/resources/dbd90dba-5916-4b14-a39c-ed0a0a66eb2d/blobs/S2B_MSIL1C_20210620T133729.nc (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2427)')))
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/Users/jan/coding/repositories/group-repo/code/cohrs/scripts/coscine_upload.py", line 17, in resource.upload("S2B_MSIL1C_20210620T133729.nc", fp, form) File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/resource.py", line 613, in upload self._upload_blob(path, handle, progress) File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/resource.py", line 695, in _upload_blob self.client.post(uri, data=monitor, headers=headers) File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/client.py", line 554, in post return self.request("POST", *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/client.py", line 463, in request return self.send_request(request, stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/client.py", line 426, in send_request self.handle_request_exception(error) File "/Users/jan/miniconda3/envs/hsu/lib/python3.11/site-packages/coscine/client.py", line 479, in handle_request_exception raise ConnectionError( ConnectionError: Failed to connect to Coscine! Check your internet connection or whether Coscine is currently down. S2B_MSIL1C_20210620T133729.nc: 65%|#######################################1 (closed) | 2.29G/3.51G [00:38<00:20, 60.0MB/s]