Shell script, curl with variable as part of url - bash

I am writing a small bash script that gets the profile of a Minecraft user from their username. However, when I run the following script, the profile_json variable comes out blank while the uuid_json and uuid variables have the content they should.
#!/bin/bash
# settings
username="Notch"
# script
uuid_json=$(curl https://api.mojang.com/users/profiles/minecraft/$username)
echo $uuid_json
uuid=$(jq '.id' <<< "$uuid_json")
echo $uuid
profile_json=$(curl https://sessionserver.mojang.com/session/minecraft/profile/$uuid)
echo $profile_json
Moreover, the output from running the script in a terminal shows the Current Speed of profile_json=$(curl https://sessionserver.mojang.com/session/minecraft/profile/$uuid) as 0. Any ideas as to why this would be?
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 56 100 56 0 0 160 0 --:--:-- --:--:-- --:--:-- 160
{"name":"Notch","id":"069a79f444e94726a5befca90e38aaf5"}
"069a79f444e94726a5befca90e38aaf5"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0

AS you can see by executing echo $uuid you got "069a79f444e94726a5befca90e38aaf5"
instead of 069a79f444e94726a5befca90e38aaf5 there should be no quotes.
jq has option to remove them it's -r with that everything works as expected(I think).
So change it to:
#!/bin/bash
# settings
username="Notch"
# script
uuid_json=$(curl https://api.mojang.com/users/profiles/minecraft/$username)
echo $uuid_json
uuid=$(jq -r '.id' <<< "$uuid_json") echo $uuid
profile_json=$(curl https://sessionserver.mojang.com/session/minecraft/profile/$uuid)
echo $profile_json

Your echo $uuid outputs this:
"069a79f444e94726a5befca90e38aaf5"
But it really should be:
069a79f444e94726a5befca90e38aaf5
for the curl command to work. So, tweak your jq line to strip off quotation marks.

Related

curl from url and rename to random uuid

i have the following command to curl a list of urls but I want to rename the output files to something else, as the original name is really long and results in the error below:
command:
jq -r '… | .[]' | xargs -I{} curl -O {}
error:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Failed to create the file
Warning: 270230751_944073806238863_7984852070388291566_n.jpg?stp=dst-jpg_e35&_n
Warning: c_ht=example.njrsnnkjnrjgngkrnngrggk&_nc_cat=111&_nc_ohc=Ch26T4U5kDIAX
Warning: 8viWsL&edm=AABBvjUBAAAA&ccb=7-4&ig_cache_key=MjczOTA2MDk4MDU3MTIyNjQ1N
0 92492 0 1 0 0 1 0 25:41:32 --:--:-- 25:41:32 1
curl: (23) Failed writing body (0 != 1)
how to pass some argument to curl so that it renames the file to a smaller name eg 16 digit uuid or something like njfnjsnf48u8 but not too common as there can be upto 1000 files being downloaded at once.
You can manipulate the {} variable that xargs uses in curl command to make it more unique. If you are sure that there are no duplicate urls in the input file then the following might work.
curl {} -o $(echo {}|base64)

Create Jira issue from Jenkins

Hi, I am trying to create a Jira issue through Jenkins on a windows slave. The console output is not showing any error, however, the Jira issue is not getting created. Below is the code:
pipeline {
agent { label 'windows'}
stages {
stage('Build') {
steps {
bat script {"""curl -u ${jira_username}:${jira_password} -X POST -H 'Content- Type:application/json' -d '{"fields":{"components":[{"id":"1"}],"fixVersions":[{"id":"2"}],"project":{"key":"KEY"},"summary":"summary","description":"description","issuetype":{"name":"Test"}}}' http://localhost:8080/rest/api/2/issue/"""}
}
}
}
}
The console output is:
D:\workspace\TestJob>curl -u username:password -X POST -H 'Content-Type:application/json' -d
'{"fields":{"components":[{"id":"1"}],"fixVersions":[{"id":"2"}],"project":
{"key":"KEY"},"summary":"summary","description":"description","issuetype":{"name":"Test"}}}'
http://localhost:8080/rest/api/2/issue/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 136 0 0 100 136 0 323 --:--:-- --:--:-- --:--:-- 326
It doesn't throw any errors but the issue is also not getting created.
However, The same code if I run on Linux slave then I get the below response and the issue is getting created.
+ curl -u 'username:password' -X POST -H Content-Type:application/json -d '{"fields":
{"components":[{"id":"1"}],"fixVersions":[{"id":"2"}],"project":
{"key":"KEY"},"summary":"summary","description":"description","issuetype":
{"name":"Test"}}}' http://localhost:8080/rest/api/2/issue/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 255 0 87 100 168 115 223 --:--:-- --:--:-- --:--:-- 223
100 255 0 87 100 168 115 223 --:--:-- --:--:-- --:--:-- 223
{"id":"648","key":"KEY-35","self":"http://localhost:8080/rest/api/2/issue/648"}
As you can see the issue is getting created on Linux but not on windows. Requesting help to resolve this issue.
Resolved the issue
Firstly 'Content-Type: application/json' should be in double-quotes.
Putting it in single quote 'Content-Type: application/json' was actually giving 415 error code which I got to know after using -k -D- in the curl command
Secondly, I had to use double backslash in the data.
The command which actually worked is:
bat script {"""curl -u username:password --header "Content-Type:application/json" -X POST --data "{\\"fields\\":{\\"project\\":{\\"key\\":\\"KEY\\"},\\"summary\\":\\"summary\\",\\"description\\":\\"description\\",\\"issuetype\\":{\\"name\\":\\"Test\\"},\\"components\\":[{\\"id\\":\\"1\\"}],\\"fixVersions\\":[{\\"id\\":\\"2\\"}]}}" http://localhost:8080/rest/api/2/issue/"""}

If elif else not working in bash ChromeOS

I am trying to make a bash script which basically takes a bunch of .debs, unpacks them and place binaries and libs in /usr/local/opt/{lib}bin.
The script checks whether / is mounted as ro or rw, and if mounted as ro to remount it as rw.
On chromebooks however, in order to mount / as rw you need to remove_rootfs_verification for the partition in question. The script fails to echo what stated above when rootfs_verification is enabled for /, and should exit 1, instead it carries on.
Here is the part of the script I am referring to
### ChromeOS's Specific!!!
# The following assumes rootfs_verification for / has already been removed
if grep $rootfs /proc/mounts | grep ro; then
mount -o remount,rw / &> mount.out
elif
grep -iw 'read-write' mount.out; then
echo '\nrootfs_verification for the root partition must to be removed in order to remount,rw /
To remove rootfs_verification run the following command and than reboot the system:
"sudo /usr/share/vboot/bin/make_dev_ssd.sh --remove_rootfs_verification --partitions 4"'
else exit 1
fi
The entire WIP script can be found here https://pastebin.com/ekEPSvYy
This is what happen when I execute it
localhost /usr/local # ./kvm_install.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 124 0 124 0 0 322 0 --:--:-- --:--:-- --:--:-- 345
100 135 100 135 0 0 170 0 --:--:-- --:--:-- --:--:-- 170
100 60384 100 60384 0 0 57950 0 0:00:01 0:00:01 --:--:-- 344k
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 143 0 143 0 0 407 0 --:--:-- --:--:-- --:--:-- 412
100 154 100 154 0 0 202 0 --:--:-- --:--:-- --:--:-- 202
100 1298k 100 1298k 0 0 929k 0 0:00:01 0:00:01 --:--:-- 3020k
/dev/root / ext2 ro,seclabel,relatime,block_validity,barrier,user_xattr,acl 0 0
./kvm_install.sh: line 31: /etc/env.d/30kvm: Read-only file system
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 66802 100 66802 0 0 69657 0 --:--:-- --:--:-- --:--:-- 74555
./kvm_install.sh: line 39: ar: command not found
tar (child): control.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
md5sum: md5sums: No such file or directory
Basically what happens here is ar cannot be found as the script was unable to add the PATH variables to /etc/env.d/30kvm since the root partition cannot be mounted because roots_verification is enabled on /.
I tried adding the elif "grep" command in [[ as some suggested here, but that didn't work and adds further syntax issues.
I am in the process of learnign the basics of bash scripting. I apologize if the script is written poorly.
Thanks
I ultimately ended up doing this.
if grep $rootfs /proc/mounts | grep 'ro,'; then
mount -o remount,rw / &> mount.out
if
grep 'read-write' mount.out; then
echo 'something to echo' && exit 1
fi
fi
It is not pretty, but it works until I find/ learn a better way to implement the loop.
to make the output of a command a variable do:
$variable="$(command)"
if you want to use grep use the syntax:
command | grep text
if you want to do if statements use the syntax:
if [ some text ]; text
commands
elif [ some text ]
commands
else
commands
fi
for the some text check this cart
grep -iw is not valid on chromebook
rootfs has a different path depending on a lot of things if you want to save it as $rootfs do this command
rootfs="$(rootdev -s)"
also you made some mistake is the "|" and "&"
command1 | command to edit command1
command1 || run this command if command1 fails
command1 && run this command if command1 succeeds

Error in bash time comparison

I have a bash script that creates a file with a timestamp as the name. Once some time passes, it is supposed to pick up that file and do something with it. I want it to pick it up after two hours, but for some reason, it is picking it up after 57 minutes (and 6 seconds). Can anyone point me to an error in my logic or assumptions?
Here are the details:
I have a variable set to 2 hours (7200 seconds):
SERVICE_DURATION=${SERVICE_DURATION:-7200} # seconds
I am setting the filename equal to the Unix timestamp concatenated with nanoseconds:
active_name=`date +%s%N`
echo "${1}" >> ${ACTIVE_DIR}/${active_name}
I then loop forever until the time is right:
while true
do
for fa in ${ACTIVE_DIR}/*
do
if [ $(basename ${fa}) -le $(($(date +%s%N) - ${SERVICE_DURATION} * 1000000000)) ]
then
exec 6< "${fa}"
read old_port <&6
read old_host <&6
read old_config <&6
exec 6<&-
logger -p daemon.info "Recycling port ${old_port}."
start_remote_service "${old_port}"
stop_remote_service "${old_port}" "${old_host}" "${old_config}" "${fa}"
sleep 2
fi
done
sleep 30
done
I can't see what is wrong with this. The filename ($(basename ${fa})) shouldn't be less than the current time minus the specified duration in nanoseconds ($(($(date +%s%N) - ${SERVICE_DURATION} * 1000000000))) until the duration has passed.
In order to keep the script from constantly checking, there is a sleep 30 at the end of the loop, so it could be that the time is somewhere between 56:36 and 57:06.
Any help would be appreciated. Thanks.

How can I quiet all the extra text when using curl within a shell script?

Here's an example ip proxy checker shell script:
#!/bin/sh
while read IP
do
CURL=$(curl -x http://$IP -L http://icanhazip.com)
echo "$CURL"
done < ip.txt
But instead of a simple result like:
0.0.0.0
1.1.1.1
I get:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
0.0.0.0
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
1.1.1.1
How can I quiet the extra stuff?
-s/--silent
Silent mode. Don't show progress meter or error messages. Makes Curl mute.
If this option is used twice, the second will again disable mute.
CURL=$(curl -s -x http://$IP -L http://icanhazip.com)

Resources