I want to take a local file and zip it at some other temp location.
Following is my code which is giving error
path="/home/tft/Downloads/ie.js"
temp="/home/tft/Downloads/Temp"
name=$(basename "$path")
echo "$path"
echo "$name"
echo "$temp"
echo "$temp/$name.zip" #Its output is also weird!
zip -r -j "$temp/$name.zip" $path
Getting below output:
/bin/bash test.sh
/home/tft/Downloads/ie.js
ie.js
/home/tft/Downloads/Temp
.zipjstft/Downloads/Temp
zip warning: name not matched: /home/tft/Downloads/ie.js
)zip . -i /home/tft/Downloads/ie.js -r -j /home/tft/Downloads/Temp
Your script is in DOS format. Convert it first to UNIX format:
sed -i 's|\r||' yourscript.sh
Or use dos2unix:
dos2unix yourscript.sh
The error messages you see is caused by having an extra character (carriage return \r) at the end of your values. This happens when your file's format is not UNIX but DOS since DOS' line endings is \r\n where UNIX only has \n.
Related
How do you use a command line argument as a file path and check for file existence in Bash?
I have the simple Bash script test.sh:
#!/bin/bash
set -e
echo "arg1=$1"
if [ ! -f "$1" ]
then
echo "File $1 does not exist."
exit 1
fi
echo "File exists!"
and in the same directory, I have a data folder containing stuff.txt.
If I run ./test.sh data/stuff.txt I see the expected output:
arg1=data/stuff.txt
"File exists!"
However, if I call this script from a second script test2.sh, in the same directory, like:
#!/bin/bash
fn="data/stuff.txt"
./test.sh $fn
I get the mangled output:
arg1=data/stuff.txt
does not exist
Why does the call work when I run it manually from a terminal, but not when I run it through another Bash script, even though both are receiving the same file path? What am I doing wrong?
Edit: The filename does not have spaces. Both scripts are executable. I'm running this on Ubuntu 18.04.
The filename was getting an extra whitespace character added to it as a result of how I was retrieving it in my second script. I didn't note this in my question, but I was retrieving the filename from folder list over SSH, like:
fn=$(ssh -t "cd /project/; ls -t data | head -n1" | head -n1)
Essentially, I wanted to get the filename of the most recent file in a directory on a remote server. Apparently, head includes the trailing newline character. I fixed it by changing it to:
fn=$(ssh -t "cd /project/; ls -t data | head -n1" | head -n1 | tr -d '\n' | tr -d '\r')
Thanks to #bigdataolddriver for hinting at the problem likely being an extra character.
I wrote a shell script that calls the ffmpeg tool but when I run it, it says No such file or directory yet it does!
Here is my script:
#!/bin/bash
MAIN_DIR="/media/sf_data/pipeline"
FFMPEG_DIR="/media/sf_data/livraison_transcripts/ffmpeg-git-20180208-64bit-static"
for file in MAIN_DIR/audio_mp3/*.mp3;
do
cp -p file FFMPEG_DIR;
done
for file in FFMPEG_DIR/*.mp3;
do
./ffmpeg -i ${file%.mp3}.ogg
sox $file -t raw --channels=1 --bits=16 --rate=16000 --encoding=signed-
integer --endian=little ${file%.ogg}.raw;
done
for file in FFMPEG_DIR/*.raw;
do
cp -p file MAIN_DIR/pipeline/audio_raw/;
done
and here is the debug response:
cp: cannot stat ‘file’: No such file or directory
./essai.sh: line 14: ./ffmpeg: No such file or directory
sox FAIL formats: can't open input file `FFMPEG_DIR/*.mp3': No such file or
directory
cp: cannot stat ‘file’: No such file or directory
FYI I'm running CentOS7 on VirtualBox
Thank you
Here's a Minimal, Complete, and Verifiable example (MCVE), a version of your script that removes everything not required to show the problem:
#!/bin/bash
MAIN_DIR="/media/sf_data/pipeline"
echo MAIN_DIR
Expected output:
/media/sf_data/pipeline
Actual output:
MAIN_DIR
This is because bash requires a $ when expanding variables:
#!/bin/bash
MAIN_DIR="/media/sf_data/pipeline"
echo "$MAIN_DIR"
The quotes are not required to fix the issue, but prevent issues with whitespaces.
Hi You need couple of correction in your shell script see below. To get the actual value assigned to a variable you need to add $ at the front of the variable in shell script.
for file in $"MAIN_DIR"/audio_mp3/*.mp3;
do
cp -p "$file" "$FFMPEG_DIR";
done
for file in "$FFMPEG_DIR"/*.mp3;
./ffmpeg -i ${file%.mp3}.ogg
#provide full path like /usr/bin/ffmpeg
for file in "$FFMPEG_DIR"/*.raw;
do
cp -p "$file" "$MAIN_DIR"/pipeline/audio_raw/;
done
I have a .txt file which has sed replace commands on each line and has 1000+ entries. The file is on my server.
Can any one please help me how to run the commands in file line by line and execute till the end of file.
My txt file looks like
sed -i 's|http://www.myoldsite/url/category/another/blah|http://www.mynewsite.com|g' ./db.sql
sed -i 's|http://www.myoldsite/url/category/blah/blah|http://www.mynewsite.com|g' ./db.sql
sed -i 's|http://www.myoldsite/url/category/blah|http://www.mynewsite.com|g' ./db.sql
You can convert your file to a list of substitution commands by removing all occurrences of sed -i ' and ' ./db.sql.
Using process substitution, the list can then be processed as a file passed to the sed -f option.
sed -i -f <(sed "s/[^']*'//;s/'.*//" file) ./db.sql
Maybe this one helps:
for i in $(cat yourfile.txt)
do
sudo $i
done
EDIT:
Why downvote?
There are multiple solutions to your problem.
SOLUTION 1
Running each command in your .txt file:
Make your .txt file executable by giving command:
chmod +x yourfile.txt
And then executing it by giving a command:
./yourfile.txt
SOLUTION 2
Creating a script
#!/bin/sh
FILE=$1
while read line; do
$line
done < $FILE
making the script executable:
chmod +x yourscriptfile.sh
And then executing your script by providing your .txt file as argument:
./yourscriptfile.sh yourfilewithcommands.txt
Hope this helps.
You can turn the file into a script by adding
#!/bin/sh
as the first line, than making the file executable with
chomd u+x file
and then just run it with
./file
BUT, this will be very slow. It's much faster to run sed only once with all the expressions, i.e. changing it into
sed -i~ 's|http://www.myoldsite/url/category/another/blah|http://www.mynewsite.com|g;
s|http://www.myoldsite/url/category/blah/blah|http://www.mynewsite.com|g;
s|http://www.myoldsite/url/category/blah|http://www.mynewsite.com|g
' ./db.sql
I'm new in ksh world and I have a problem right now with a script. The script under this lines is into the .profile file of a user in a UNIX machine and when I try to connect whith him i get always the error
home/userTest/.profile: syntax error: `if' unmatched
I don't know how to solve this, because I suppose that this scripts defines the prompt for the connected user, and if I have this error the prompt only shows "$"
I tried the command
ksh -n /home/userTest/.profile
and I get the error always in the last line of the file
#!/bin/ksh
# ksh example
if [[$0 = "ksh"]];
then
bash
exit $?
fi
if [[$0 = "-ksh"]];
then
bash --login
exit $?
fi
export LOGIN=$LOGNAME
#prompt config
PS1="$LOGIN#"$(hostname)":$PWD"
if [["$(id -u)" = "0"]];
then
export PS1="$PS1# "
else
export PS1="$PS1> "
fi
#Alias utile
alias ll="ls -la"
#Set any export here
export PATH_EXAMPLE=/home/userTest
export JAVA_HOME=$PATH_EXAMPLE/games/java/current
export PATH=$JAVA_HOME/bin:$PATH
How can I solve this problem ?
Thanks.
I had the same error. Turned out it was due to DOS format newlines (CR-LF) in my *.sh file created in Windows and then transferred to a Linux server.
Commands to convert DOS format newlines (CR-LF) to UNIX format newlines (LF)
In Windows: using Notepad++, as explained here:
From the "Edit" menu, select "EOL Conversion" -> "UNIX/OSX Format".
You can also set the default EOL in notepad++ via "Settings" -> "Preferences" -> "New Document/Default Directory" then select "Unix/OSX" under the Format box.
In UNIX/Linux: using one of the techniques explained here:
Convert DOS to UNIX using sed command:
sed 's/^M$//' input.txt > output.txt
Convert DOS to UNIX using tr command:
tr -d '\r' < input.file > output.file
Convert DOS to UNIX using this Perl one-liner:
perl -pi -e 's/\r\n/\n/g' input.file
Convert DOS to UNIX using dos2unix command:
dos2unix myfile.txt or dos2unix -b myfile.txt (with a backup)
[ Bonus tip ]
Commands to convert UNIX format newlines (LF) to DOS format newlines (CR-LF)
Convert UNIX to DOS using unix2dos command:
unix2dos myfile.txt or unix2dos -b myfile.txt (with a backup)
Convert UNIX to DOS using sed command:
sed 's/$'"/`echo \\\n\\\r`/" input.txt > output.txt (you need those \\\, you do)
I am using following version
version sh (AT&T Research) 93u+ 2012-08-01
I did not received any syntax error for your above code , though there a problem with your if statement condition instead of
if [[$0 = "-ksh"]]
it should be
if [[ $0 == "-ksh" ]]
or
if [[ $0 = "-ksh" ]]
the latter is obsolete
The complete code is as below
#!/bin/ksh
# ksh example
if [[ $0 = "ksh" ]];
then
bash
exit $?
fi
if [[ $0 == "-ksh" ]];
then
bash --login
exit $?
fi
export LOGIN=$LOGNAME
#prompt config
PS1="$LOGIN#"$(hostname)":$PWD"
if [[ "$(id -u)" == "0" ]];
then
export PS1="$PS1# "
else
export PS1="$PS1> "
fi
#Alias utile
alias ll="ls -la"
#Set any export here
export PATH_EXAMPLE=/home/userTest
export JAVA_HOME=$PATH_EXAMPLE/games/java/current
export PATH=$JAVA_HOME/bin:$PATH
You script may be having some unwanted character , try to look out for then using cat -vte
you can also try command dos2unix filename and then run ksh -n
Im currently using 3 different commands to achieve my goal of downloading a zip, extracting it, converting the txt file to utf8 and then converting the csv to json!
First I have:
wget https://www.example.com/example.zip -O temp.zip; unzip -o temp.zip; rm temp.zip
Which is good, but the problem to start with is how do I rename the file that is extracted so its the same every time for the next processes as it can be a different name within the zip every day? Next I run this script depending on the filename that converts the ISO to utf8:
sh dir_iconv.sh example1.txt ISO8859-1 UTF-8
Which is this script:
#!/bin/bash
ICONVBIN='/usr/bin/iconv' # path to iconv binary
if [ $# -lt 3 ]
then
echo "$0 dir from_charset to_charset"
exit
fi
for f in $1/*
do
if test -f $f
then
echo -e "\nConverting $f"
/bin/mv $f $f.old
$ICONVBIN -f $2 -t $3 $f.old > $f
rm -f $f.old
else
echo -e "\nSkipping $f - not a regular file";
fi
done
And then finally I run a ruby script csv2json - https://github.com/darwin/csv2json - that is called as follows (pipe delimited) to give me a json output:
csv2json -s '|' example1.txt > example1.json
Is there a simple way to roll this into one command or script that can be called?
Pipe all your commands one after another and, if necessary, throw them in a shell script file.
wget -qO- https://www.example.com/example.zip | unzip | iconv -f ISO8859-1 -t UTF-8 | csv2json > example.json