I have this bash script code:
#!/bin/bash
FILES=/home/user/Downloads/asd/
for f in $FILES
do
curl -F dir="#/home/user/Downloads/asd;$f" -F Match=3 -F "Name=DrKla" \
-F countNo=1 -F outputFormat=json "http://somelink.com"
done
Inside the asd folder there are 6 files and I want them to be uploaded 1 by 1 with this code as an argument of -F "dir=#...."
When I run my code I get the error:
Warning: skip unknown form field: /home/user/Downloads/asd/
curl: (43) A libcurl function was given a bad argument
Here is a working version of the code for a single file:
curl -F dir="#/home/user/Downloads/asd/count.txt" -F Match=3 -F "Name=DrKla" \
-F countNo=1 -F outputFormat=json "http://somelink.com"
So I want to have all the files in asd folder to be read and uploaded like this. I don't see what's wrong with my do loop.
The issues appear to be that you only give a path, not a reference to all files in the path * and there is a strange semi-colon ; in your path:
#!/bin/bash
FILES=/home/user/Downloads/asd/*
for f in $FILES
do
curl -F dir="#$f" -F Match=3 -F "Name=DrKla" \
-F countNo=1 -F outputFormat=json "http://somelink.com"
done
I'm not sure what the # is for or if it is needed, but $f should already contain the path.
Related
I am trying to create a pipeline in gitlab-ci. my problem is that when I run ls command I can see my files, but when I use for condition, I cannot find same files. any suggestion?
script:
- set -vux
- ls ./dist/*
- |
for file in ./dist/*; do
if [ -f ${file} ]; then
export ARTIFACTORY_IMAGE_TAG=${DATASCIENCE_REPO}/$(basename ${file})/$IMAGE_VERSION
'curl -H "X-JFrog-Art-Api: $ARTIFACTORY_PW" -XPUT "$ARTIFACTORY_IMAGE_TAG" -T dist/$file'
fi
done
dependencies:
- package-build
my output is like this:
$ ls ./dist/*
ls ./dist/*
++ ls ./dist/__init__.py ./dist/my_proj-0.0.1.1164649-py3-none-any.whl ./dist/file1.py ./dist/file2.py ./dist/file3.json ./dist/file4.json
./dist/__init__.py
./dist/my_proj-0.0.1.1164649-py3-none-any.whl
./dist/file1.py
./dist/file2.py
./dist/file3.json
./dist/file4.json
echo $'\x1b[32;1m$ for file in ./dist/*; do # collapsed multi-line command\x1b[0;m'
++ echo '$ for file in ./dist/*; do # collapsed multi-line command'
$ for file in ./dist/*; do # collapsed multi-line command
for file in ./dist/*; do
if [ -f ${file} ]; then
export ARTIFACTORY_IMAGE_TAG=${MY_REPO}/$(basename ${file})/$IMAGE_VERSION
'curl -H "X-JFrog-Art-Api: $ARTIFACTORY_PW" -XPUT "$ARTIFACTORY_IMAGE_TAG" -T dist/$file'
fi
done
++ for file in ./dist/*
++ '[' -f ./dist/__init__.py ']'
+++ basename ./dist/__init__.py
++ export ARTIFACTORY_IMAGE_TAG=https://artifactory.xyz.com/artifactory/my_proj_dev/__init__.py/0.0.1.1164649
++ ARTIFACTORY_IMAGE_TAG=https://artifactory.xyz.com/artifactory/my_proj_dev/__init__.py/0.0.1.1164649
++ 'curl -H "X-JFrog-Art-Api: $ARTIFACTORY_PW" -XPUT "$ARTIFACTORY_IMAGE_TAG" -T dist/$file'
/scripts-11718-6374270/step_script: line 239: curl -H "X-JFrog-Art-Api: $ARTIFACTORY_PW" -XPUT "$ARTIFACTORY_IMAGE_TAG" -T dist/$file: No such file or directory
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: command terminated with exit code 1
Update 1: according to the error I am receiving, I think in my curl command system cannot file the file. but why I am not sure?
Simple syntax mistake.
The error says there is No such file or directory.
line 239: curl -H "X-JFrog-Art-Api: $ARTIFACTORY_PW" -XPUT "$ARTIFACTORY_IMAGE_TAG" -T dist/$file: No such file or directory
An experiment.
Let's run these three commands in your terminal.
The first one
(DO NOT REMOVE A SINGLE QUOTE.)
'touch love and peace'
The result is in bash
-bash: touch love and peace: command not found
The result is in zsh
zsh: command not found: touch love and peace
A terminal recognizes the single-quoted string as a command in the first command.
The second one
(DO NOT REMOVE A SINGLE QUOTE.)
'touch /'
The result is in bash
-bash: touch /: No such file or directory
The result is in zsh
zsh: no such file or directory: touch /
A terminal recognizes the single-quoted string with the slash as a file in the second command.
The third one
without a single quote.
touch love and peace
The result is
$ls
and love peace
If you intend to run the curl command in every loops,
Just remove the single quotes.
curl -H "X-JFrog-Art-Api: $ARTIFACTORY_PW" -XPUT "$ARTIFACTORY_IMAGE_TAG" -T $file
The solution
for file in ./dist/*; do
if [ -f ${file} ]; then
export ARTIFACTORY_IMAGE_TAG=${DATASCIENCE_REPO}/$(basename ${file})/$IMAGE_VERSION
curl -H "X-JFrog-Art-Api: $ARTIFACTORY_PW" -XPUT "$ARTIFACTORY_IMAGE_TAG" -T $file
fi
done
It seems like the problem is that you are not listing the files in your for statement. Try:
for file in `ls ./dist/*`; do
echo "act on $file"
done
I need to do a curl upload with a file where I don't know the exact file name.
"curl
-F \"status=2\"
-F \"notify=1\"
-F \"ipa=#${FILE}\"
-F \"teams=${TEAM_ID}\"
-H \"X-HockeyAppToken: ${APITOKEN}\"
https://rink.hockeyapp.net/api/2/apps/${APPVERSION}/app_versions/upload"
This is done in gitlab-ci and the FILE variable is set to build/com.test.app_v*.ipa. The file I want to upload has a version number set and has the path build/com.test.app_v1.0.0.0.ipa. The problem I have now is that the * does not get expanded inside this curl call. I've tried it with an export before:
- export ABSOLUTE_FILENAME=${FILE}
- "curl
-F \"status=2\"
-F \"notify=1\"
-F \"ipa=#${ABSOLUTE_FILENAME}\"
-F \"teams=${TEAM_ID}\"
-H \"X-HockeyAppToken: ${APITOKEN}\"
https://rink.hockeyapp.net/api/2/apps/${APPVERSION}/app_versions/upload"
Still I'm gettin an error curl: (26) couldn't open file "build/com.test.app_v*.ipa" How can I expand the path to an absolute path before my curl upload?
With realpath command:
...
-F \"ipa=#$(realpath $FILE)\"
...
I want to make a mailgun curl call using windows batch file. Since windows shell doesn't support multiple lines, how can I execute the below curl function in windows batch file?
curl -s --user 'api:key-xxxxxxxxxx' \
https://api.mailgun.net/v3/sandboxbxxxxxxxxxxxxx.mailgun.org/messages \
-F from='user <email#gmail.com>' \
-F to='user <email#live.com>' \
-F subject='Hello' \
-F text='body!' \
-F attachment=#test.txt \
Update
When I tried to execute the command after removing the multiple lines it returned this error:
curl -s --user 'api:key-xxxxxxxxxx' https://api.mailgun.net/v3/sandboxbxxxxxxxxxxxxx.mailgun.org/messages -F from='user -F to='user -F subject='Hello' -F text='body!' -F attachment=#test.txt 0<email#live.com 1>'
The system cannot find the file specified.
PS: The attachment file is in the same directory
Thanks!
simply on one line and put the <> redirection char between " or escape it with ^:
curl -s --user 'api:key-xxxxxxxxxx' https://api.mailgun.net/v3/sandboxbxxxxxxxxxxxxx.mailgun.org/messages -F from="user <email#gmail.com>" -F to="user <email#live.com>" -F subject='Hello' -F text='body!' -F attachment=#test.txt
You can also create variable for each element :
set "$ApiKey=api:key-xxxxxxxxxx"
set "$Url=https://api.mailgun.net/v3/sandboxbxxxxxxxxxxxxx.mailgun.org/messages"
set "$From=email#gmail.com"
....
and then
curl -s --user '%$ApiKey%' %$Url% -F from="user <%$From%>" -F to= ....
I would like to implement this as a Makefile task:
# step 1:
curl -u username:password -X POST \
-d '{"name": "new_file.jpg","size": 114034,"description": "Latest release","content_type": "text/plain"}' \
https://api.github.com/repos/:user/:repo/downloads
# step 2:
curl -u username:password \
-F "key=downloads/octocat/Hello-World/new_file.jpg" \
-F "acl=public-read" \
-F "success_action_status=201" \
-F "Filename=new_file.jpg" \
-F "AWSAccessKeyId=1ABCDEF..." \
-F "Policy=ewogIC..." \
-F "Signature=mwnF..." \
-F "Content-Type=image/jpeg" \
-F "file=#new_file.jpg" \
https://github.s3.amazonaws.com/
In the first part however, I need to get the file size (and content type if it's easy, not required though), so some variable:
{"name": "new_file.jpg","size": $(FILE_SIZE),"description": "Latest release","content_type": "text/plain"}
I tried this but it doesn't work (Mac 10.6.7):
$(shell du path/to/file.js | awk '{print $1}')
Any ideas how to accomplish this?
If you have GNU coreutils:
FILE_SIZE=$(stat -L -c %s $filename)
The -L tells it to follow symlinks; without it, if $filename is a symlink it will give you the size of the symlink rather than the size of the target file.
The MacOS stat equivalent appears to be:
FILE_SIZE=$(stat -L -f %z)
but I haven't been able to try it. (I've written this as a shell command, not a make command.) You may also find the -s option useful:
Display information in "shell output", suitable for initializing variables.
For reference, an alternative method is using du with -b bytes output and -s for summary only. Then cut to only keep the first element of the return string
FILE_SIZE=$(du -sb $filename | cut -f1)
This should return the same result in bytes as #Keith Thompson answer, but will also work for full directory sizes.
Extra: I usually use a macro for this.
define sizeof
$$(du -sb \
$(1) \
| cut -f1 )
endef
Which can then be called like,
$(call sizeof,$filename_or_dirname)
I think this is a case where parsing the output of ls is legitimate:
% FILE_SIZE=`ls -l $filename | awk '{print $5}'`
(no it's not: use stat, as noted by Keith Thompson)
For the type, you can use
% FILE_TYPE=`file --mime-type --brief $filename`
I use this command to download a series of images:
curl -O --max-time 10 --retry 3 --retry-delay 1 http://site.com/image[0-100].jpg
Some images are corrupted, so I delete them.
for i in *.jpg; do jpeginfo -c $i || rm $i; done
How to tell curl to check file existence before download?
I can use this command to prevent curl override existing images:
chmod 000 *.jpg
But I don't want to re-download them.
If the target resource is static, curl has an option -z to only download a newer target copy.
Usage example:
curl -z image0.jpg http://site.com/image0.jpg
An example for your case:
for i in $(seq 0 100); do curl -z image$i.jpg -O --max-time 10 --retry 3 --retry-delay 1 http://site.com/image$i.jpg; done
No idea about doing it with curl, but you could check it with Bash before you run the curl command.
for FILE in FILE1 FILE2 …
do
if [[ ! -e $FILE ]]; then
# Curl command to download the image.
fi
done