Does curl -C, --continue-at work when piping standard out? - bash

Question

From curl's manpage
Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.
So if using
curl \
--retry 9999 \
--continue-at - \
https://mydomain.test/some.file.bin \
| target-program
and the download fails (once) half-way through, and the server supports range requests, will curl retry, via a range request, so target-program receives the full bytes of some.file.bin as its input?

Answer 1

From testing, curl will not retry using a range request.
I wrote a broken HTTP server, requiring the client to retry using a range-request to get a full response. Using wget
wget -O - http://127.0.0.1:8888/ | less
results in the full response
abcdefghijklmnopqrstuvwxyz
and I can see on the server side there way a request with 'Range': 'bytes=24-' in the request headers.
However, using curl
curl --retry 9999 --continue-at - http://127.0.0.1:8888/ | less
results in only the incomplete response, and no range request in the server log.
abcdefghijklmnopqrstuvwx
The Python server used
import asyncio
import re
from aiohttp import web
async def main():
data = b'abcdefghijklmnopqrstuvwxyz'
async def handle(request):
print(request.headers)
# A too-short response with an exception that will close the
# connection, so the client should retry
if 'Range' not in request.headers:
start = 0
end = len(data) - 2
data_to_send = data[start:end]
headers = {
'Content-Length': str(len(data)),
'Accept-Ranges': 'bytes',
}
print('Sending headers', headers)
print('Sending data', data_to_send)
response = web.StreamResponse(
headers=headers,
status=200,
)
await response.prepare(request)
await response.write(data_to_send)
raise Exception()
# Any range request
match = re.match(r'^bytes=(?P<start>\d+)-(?P<end>\d+)?$', request.headers['Range'])
start = int(match['start'])
end = \
int(match['end']) + 1 if match['end'] else \
len(data)
data_to_send = data[start:end + 1]
headers = {
'Content-Range': 'bytes {}-{}/{}'.format(start, end - 1, len(data)),
'Content-Length': str(len(data_to_send)),
}
print('Sending headers', headers)
print('Sending data', data_to_send)
response = web.StreamResponse(
headers=headers,
status=206
)
await response.prepare(request)
await response.write(data_to_send)
await response.write_eof()
return response
app = web.Application()
app.add_routes([web.get(r'/', handle)])
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, '0.0.0.0', 8888)
await site.start()
await asyncio.Future()
asyncio.run(main())