I am trying to rename the filenames in remote server like filename-dirname.suffix
and copy the files to my server .
I had written code like ....
#!/usr/bin/bash
TRANSFERSERVERXMLS="/emp/transfer/XMLS"
REMOTESERVERXMLS="remoteemp/empdir/XMLS"
# renaming the filenames in remote server like filename-dirname.suffix
ssh abc#xyz REMOTESERVERXMLS=$REMOTESERVERXMLS 'bash -s'<< 'EOF'
for i in $REMOTESERVERXMLS/* ; do
if [[ -d $i ]]; then
dirname=$(basename $i)
for j in $REMOTESERVERXMLS/$dirname/* ; do
fname="$(basename "$j")"
prefix=$(echo $fname | awk -F "." 'NF{NF-=1};1')
suffix=$(echo $fname | awk -F "." '{print $NF}')
target=$prefix-$dirname.$suffix
mv $REMOTESERVERXMLS/$dirname/"$fname" $REMOTESERVERXMLS/$dirname/"${target// /_}"
done
fi
done
EOF
scp abc#xyz:${REMOTESERVERXMLS}/*/* ${TRANSFERSERVERXMLS}
Getting an error : EOF:Command not found
and scp is not working ( not able to copy into calling server)
You have a space before the delimiter EOF. Do not indent EOF at the end of your "here document". The delimiter (EOF) must be the only thing on the line, with no leading or trailing whitespace.
Alternatively use <<- 'EOF' and indent with a tab.
Related
so the file contains data like
# entertainment
youtube.com
twitch.tv
# research
google.com
wikipedia.com
...
and I would like to pass that file as an argument in a script that would open all lines if they doesn't start with an #. Any clues on how to ?
so far what i have:
for Line in $Lines
do
case "# " in $Line start firefox $Line;; esac
done
some code that could be useful (?):
while read line; do chmod 755 "$line"; done < file.txt
grep -e '^[^#]' inputfile.txt | xargs -d '\n' firefox --new-tab
grep -e '^[^#]': Will print all lines that don't start with a sharp (comments)
xargs -d '\n' firefox --new-tab: Will pass each line that is not blank, as argument to Firefox.
Removes both the lines that start with # and empty lines.
#!/bin/bash
#
while read -r line
do
if [[ $(echo "$line" | grep -Ev "^#|^$") ]]
then
firefox --new-tab "$url" &
fi
done <file.txt
Skip the empty lines and the lines that starts with a #
#!/usr/bin/env bash
while IFS= read -r url; do
[[ "$url" == \#* || -z "$url" ]] && continue
firefox --new-tab "$url" &
done < file.txt
awk 'NF && $1!="#"{print "firefox --new-tab", $0, "&"}' file.txt | bash
i'm using git bash for windows (git version 2.18.0.windows.1) to write a bash script like this one:
file=$1
if [[ ! -s "$file" ]] ; then #empty
echo -e "${RED}Invalid argument. Pass the file with all GCIDs as INPUT!!!${NOCOLOR}"
else
number=$(cat $file | wc -l )
number=$(($number+1))
echo -e "** ${number} GCID detected **"
echo ""
while read -r gcidRead
do
gcid=${gcidRead}
echo -e "select distinct operation from audit_trail.audit_trail where gcid='$gcid';" >> query.txt
value=$(psql "host=XXXX port=62013 dbname=prodemeagcdm user=XXXX password=XXXX" <<-EOF
select distinct operation from audit_trail.audit_trail where gcid='$gcid';
\q
EOF
)
echo -e "${value}" >> output.txt
if grep -q delete_bupa output.txt ; then
echo -e "${gcid}" >> gcidDeleted.txt
fi
done < $file
fi
I created just to debug the query.txt file in which the output is:
';lect distinct operation from audit_trail.audit_trail where gcid='XXX
instead of
select distinct operation from audit_trail.audit_trail where gcid='XXX'
In short, every string after $gcid parameter will be written at the beginning of the entire string.
If I use a unix terminal the echo output is ok.
Why in git bash the "echo" command has the wrong output mentioned?
Thanks in advance
I think you see the output on terminal of a string which contains a CR (13 in dec or 0D in hex) : the last '; cahracters are written from the beginning of the line, thus overwriting the first two characters of the string.
The string actually consists of select dist... gcid='XXX\r';, and is just printed awkwardly (to a human) on the terminal.
There are many ways to drop CR from the input, here are two of them :
# remove all CR chars from the input :
cat $file | tr -d '\r' | while read -r gcdiRead; do
...
done
# remove all CR chars only at end of lines (e.g : when followed by LF) :
cat $file | sed -e 's/\r$//' | while read -r gcdiRead; do
...
done
Content of file is:
#data.conf
ip=127.0.0.1
port=7890
delay=10
key=1.2.3.4
debug=true
Shell Script:
#!/bin/bash
typeset -A config
config=()
config_file_path="./data.conf"
cmd="java -jar ./myprogram.jar"
#This section will read file and put content in config variable
while read line
do
#echo "$line"
if echo $line | grep -F = &>/dev/null
then
key=$(echo "$line" | cut -d '=' -f 1)
config[$key]=$(echo "$line" | cut -d '=' -f 2)
echo "$key" "${config["$key"]}"
fi
done < "$config_file_path"
cmd="$cmd -lh ${config["ip"]} -lp ${config["port"]} -u ${config["debug"]} -hah \"${config["key"]}\" -hap ${config["delay"]}"
echo $cmd
Expected output:
java -jar myprogram.jar -lh 127.0.0.1 -lp 7890 -u true -hah "1.2.3.4" -hap 10 -b
Output:
Every time some unexpected o/p
Ex. -lp 7890rogram.jar
Looks like it is overwriting same line again and again
In respect to the comments given and to have an additional automatic data cleansing within the script, you could have according How to convert DOS/Windows newline (CRLF) to Unix newline (LF) in a Bash script? and Remove carriage return in Unix
# This section will clean the input config file
sed -i 's/\r$//' "${config_file_path}"
within your script. This will prevent the error in future runs.
I have a file with the following entries:
foop07_bar2_20190423152612.zip
foop07_bar1_20190423153115.zip
foop08_bar2_20190423152612.zip
foop08_bar1_20190423153115.zip
where
foop0* = host
bar* = fp
I would like to read the file and create 3 variables, the whole file name, host and fp (which stands for file_path_differentiator).
I am using read to take the first line and get my whole file name variable, I though I could then feed this into awk to grab the next two variables, however the first method of variable insertion creates an error and the second gives me all the variables.
I would like to loop each line, as I wish to use these variables to ssh to the host and grab the file
#!/bin/bash
while read -r FILE
do
echo ${FILE}
host=`awk 'BEGIN { FS = "_" } ; { print $1 }'<<<<"$FILE"`
echo ${host}
path=`awk -v var="${FILE}" 'BEGIN { FS = "_" } ; { print $2 }'`
echo ${path}
done <zips_not_received.csv
Expected Result
foop07_bar2_20190423152612.zip
foop07
bar2
foop07_bar1_20190423153115.zip
foop07
bar1
Actual Result
foop07_bar2_20190423152612.zip
/ : No such file or directoryfoop07_bar2_20190423152612.zip
bar2 bar1 bar2 bar1
You can do this alone with bash, without using any external tool.
while read -r file; do
[[ $file =~ (.*)_(.*)_.*\.zip ]] || { echo "invalid file name"; exit 1; }
host="${BASH_REMATCH[1]}"
path="${BASH_REMATCH[2]}"
echo "$file"
echo "$host"
echo "$path"
done < zips_not_received.csv
typical...
Managed to work a solution after posting...
#!/bin/bash
while read -r FILE
do
echo ${FILE}
host=`echo "$FILE" | awk -F"_" '{print $1}'`
echo $host
path=`echo "$FILE" | awk -F"_" '{print $2}'`
echo ${path}
done <zips_not_received.csv
not sure on the elegance or its correctness as i am using echo to create variable...but i have it working..
Assuming there is no space or _ in your "file name" that are part of the host or path
just separate line before with sed, awk, ... if using default space separator (or use _ as argument separator in batch). I add the remove of empty line value as basic security seeing your sample.
sed 's/_/ /g;/[[:blank:]]\{1,\}/d' zips_not_received.csv \
| while read host path Ignored
do
echo "${host}"
echo "${path}"
done
I am trying to rename the filenames in remote server like filename-dirname.suffix
and copy the files to my server .
I had written code like ....
#!/usr/bin/bash
TRANSFERSERVERXMLS="/emp/transfer/XMLS"
REMOTESERVERXMLS="remoteemp/empdir/XMLS"
# renaming the filenames in remote server like filename-dirname.suffix
ssh abc#xyz REMOTESERVERXMLS=$REMOTESERVERXMLS 'bash -s'<< 'EOF'
for i in $REMOTESERVERXMLS/* ; do
if [[ -d $i ]]; then
dirname=$(basename $i)
for j in $REMOTESERVERXMLS/$dirname/* ; do
fname="$(basename "$j")"
prefix=$(echo $fname | awk -F "." 'NF{NF-=1};1')
suffix=$(echo $fname | awk -F "." '{print $NF}')
target=$prefix-$dirname.$suffix
mv $REMOTESERVERXMLS/$dirname/"$fname" $REMOTESERVERXMLS/$dirname/"${target// /_}"
done
fi
done
EOF
scp abc#xyz:${REMOTESERVERXMLS}/*/* ${TRANSFERSERVERXMLS}
Getting an error : EOF:Command not found
and scp is not working ( not able to copy into calling server)
You have a space before the delimiter EOF. Do not indent EOF at the end of your "here document". The delimiter (EOF) must be the only thing on the line, with no leading or trailing whitespace.
Alternatively use <<- 'EOF' and indent with a tab.