I am trying to curl a long URL like the following and save it into a file:
https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ%3D%3D.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603
where the filename should be only 277901819_162489612877754_7456098591865793913_n.jpg
curl "https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ%3D%3D.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603" doesnt save it to a file
Okay, so you have tried the -o and -O options, but they don't seem to do what you want.
$ curl -o 'https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ%3D%3D.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603'
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
This fails because -o requires you to specify the filename yourself. The correct way to use -o is to specify the filename yourself:
$ curl -o '277901819_162489612877754_7456098591865793913_n.jpg' 'https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ%3D%3D.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 175k 100 175k 0 0 85047 0 0:00:02 0:00:02 --:--:-- 85527
You haven't really explained why this isn't satisfactory, so I'll assume that you are getting the URL from user input or a file or something.
This is what the -O option is for. But it turns up another problem:
$ curl -O 'https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ%3D%3D.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0Warning: Failed to create the file
Warning: 277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_n
Warning: c_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX
Warning: -prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyO
Warning: Q%3D%3D.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&
Warning: oe=6266DFF7&_nc_sid=83d603: File name too long
0 175k 0 1492 0 0 1406 0 0:02:07 0:00:01 0:02:06 1426
curl: (23) Failure writing output to destination
Depending on your system, perhaps this succeeded, but the filename wasn't what you wanted. This is trying to include the entire query string in the filename.
This seems about all you can do with curl as it stands. You either have to determine the filename yourself (such as with sed) and pass it with the -o option, or you need to use something other than curl.
I note that HTTPie handles this like you'd hope. Perhaps you could use it instead of curl?
$ http --download 'https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ%3D%3D.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603'
HTTP/1.1 200 OK
Cache-Control: max-age=1209600, no-transform
Connection: keep-alive
Content-Length: 179351
Content-Type: image/jpeg
Date: Thu, 21 Apr 2022 12:25:45 GMT
Last-Modified: Wed, 06 Apr 2022 06:31:15 GMT
X-FB-Server-Cluster-Forwarded: syd2c01
X-Frame-Options: DENY
content-digest: adler32=3445021316
cross-origin-resource-policy: same-origin
timing-allow-origin: *
x-haystack-needlechecksum: 1987900843
x-needle-checksum: 1431545784
Downloading 175.15 kB to "277901819_162489612877754_7456098591865793913_n.jpg"
Done. 175.15 kB in 1.08534s (161.38 kB/s)
You curl command is missing quote, it should be:
curl -o a.jpg 'https://instagram.fwlg2-1.fna.fbcdn.net/v/t51.2885-15/277901819_162489612877754_7456098591865793913_n.jpg?stp=dst-jpg_e35&_nc_ht=instagram.fwlg2-1.fna.fbcdn.net&_nc_cat=102&_nc_ohc=gKuf2i9Vj5MAX-prxmR&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjgxMDI0MDUzNDIyMTEzNzMyOQ%3D%3D.2-ccb7-4&oh=00_AT9BhJfQrBULLiNNXWfLQjhog3T8szmLRvMUEfWqE-cQIQ&oe=6266DFF7&_nc_sid=83d603'
Related
I'm using CURL to get StatusCode at Windows Putty on Linux.
When I try with:
curl -k -I <url>
it gives 200 which is fine:
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Thu, 01 Jan 1970 00:00:01 GMT
Content-Length: 0
Content-Language: en-US
but when I try code below to get only StatusCode number, it gives 000:
curl -o /dev/null --silent --head --write-out '%{http_code}\n' <url>
response: 000
What is the issue here?
cURL 000 means server did not respond. It can be due to Firewall blockage, or you are using on localhost or ca issue.
I use curl commands in my jenkins jobs and I have some questions about them. I have a command like this. Here a get a json object, parse it and put the result to array "fileslist"
fileslist=($(curl -u user:password -X POST -k http://server:8081/artifactory/api/search/aql -d "items.find({\"type\" : \"folder\", \"repo\" : \"${REPOSITORY}\", \"path\" : \".\", \"name\" : {\"\$nmatch\" : \"\.\"}, \"modified\" : {\"\$lt\" : \"$borderdate\"} })" | jq --raw-output '.results | .[] | .name'))
When I use it, I have this information in my output , that I don't really want to see.
.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 1196 100 1052 100 144 227k 31858 --:--:-- --:--:-- --:--:-- 1027k
I can use --silent option, but it makes silent all the output, so it doesn't match with --fail option. What I need is to see in my log only information about errors, that occurred during the curl command.
I want the command above (if something goes wrong) write a error and STOP a Jenkins job. When I use --fail option, it just writes to the output about a error and my script continues. You can tell me to use Parse Input Plugin, but it is not what I want: It marks a build as "failed build" AFTER the script finished it's work, but I want to make it stop immediately.
The same questions about wget command.
Thank you in advance!
When I run a curl command against my non-running server, I get below output.
I only want
Failed to connect: Connection Refused
How can I remove the extra output. If I run --silent, it suppress everything.
I get this o/p for curl http://localhost:300/index.html
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 9626 100 9626 0 0 148k 0 --:--:-- --:--:-- --:--:-- 149k
Failed to connect: Connection Refused
yes! using -s, --silent along with -S, --show-error is the right option.
Curl gives me the compressed output by default, but try:
curl --compressed http://localhost:300/index.html
By contrast --verbose returns more lengthy output.
man curl will show you all the options available to you.
Looks like I need to use -sS. man curl will show you all the options available to you.
I want to download a binary from https://github.com/sschwartzman/newrelic-unix-plugin. but the url will redirect to another address, so I use the awk to parse it. e.g.
curl -I https://github.com/sschwartzman/newrelic-unix-plugin/blob/master/dist/newrelic_unix_plugin.tar.gz\?raw\=true | awk '/Location:/ {print $2}'
And the result is as I expected, e.g.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
https://github.com/sschwartzman/newrelic-unix-plugin/raw/master/dist/newrelic_unix_plugin.tar.gz
The real address is parsed. so I just want to use curl to download it. but I always got fail.
My os is osx El Capitan (I don't have wget).
curl -I https://github.com/sschwartzman/newrelic-unix-plugin/blob/master/dist/newrelic_unix_plugin.tar.gz\?raw\=true | awk '/Location:/ {curl -O $2}'
p.s.
I tried to download the binary directly by curl, but still failed. The downloaded the file was incorrect size and content.
curl -O https://github.com/sschwartzman/newrelic-unix-plugin/raw/master/dist/newrelic_unix_plugin.tar.gz
curl has switch -L which can handle redirect.
So, this should work:
curl -L https://github.com/sschwartzman/newrelic-unix-plugin/blob/master/dist/newrelic_unix_plugin.tar.gz\?raw\=true -o newrelic_unix_plugin.tar.gz
I have a ruby script:
#!/usr/bin/env ruby
`curl -X GET http://host/someurl'
The response doesnt get displayed on terminal when I run this script:
$ ./script.rb
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 146 0 146 0 0 73 0 --:--:-- 0:00:01 --:--:-- 73
$
The server does send some data. If I supply -o to curl:
`curl -X GET -o <some_file> http://host/someurl'
some_file contains server response. Same works for POST requests though:
`curl -X POST --data-binary #some_file http://host/someurl'
This shows the response on terminal. Any idea how I fix this?
just puts it
puts `curl -X GET http://host/someurl`
You have a typo closing your back-tick shell command. You want:
`curl -X GET http://host/someurl`
You used a single quotation mark instead of a trailing back-tick. This causes the expression not to be terminated.