I just got into bash so I decided to write my first script. To give you a little background I want to write a script that will back up my Documents folder onto a USB stick whenever I connect the USB stick. (I am aware that software like this exists).
I have two strings at the beginning of the script:
directoryPath="/Users/USER/Documents" # Folder I want to backup
backupPath="/Volumes/backMeUp/main" # Where I want the folder to backup
For loops gives me absolute path to a file like this
/Users/USER/Documents/Comics/Bleach/Volume 004/bleach_031.zip
Until now I was using sed like this
newPath=`echo "$file" | sed "/Users\/USER\/Documents/s//Volumes\/backMeUp\/main/"`
But since I want my script to be more "open" and other-user-friendly I want to get rid of this line and make it some other way.
I also tried this with different syntax
echo $file | sed -e "s/$directoryPath/$backupPath/"
but with no luck.
My question is how can I remove part of a string with $directoryPath and replace it with $backupPath?
basename (and dirname) are your friend(s).
Something like this:
#!/bin/bash
directoryPath="/Users/rojcyk/Documents" # Folder I want to backup
backupPath="/Volumes/backMeUp/main" # Where I want the folder to backup
f=$(basename '/Users/rojcyk/Documents/Comics/Bleach/Volume 004/bleach_031.zip')
echo ${backupPath}/${f}
Updated
#!/bin/bash
directoryPath="/Users/rojcyk/Documents" # Folder I want to backup
backupPath="/Volumes/backMeUp/main" # Where I want the folder to backup
f='/Users/rojcyk/Documents/Comics/Bleach/Volume 004/bleach_031.zip'
# delete shortest match of substring from front of string
new_f=${f#$directoryPath}
echo ${backupPath}${new_f}
Output:
/Volumes/backMeUp/main/Comics/Bleach/Volume 004/bleach_031.zip
Read more about bash string operations here
With bash:
directoryPath="/Users/rojcyk/Documents"
backupPath="/Volumes/backMeUp/main"
f="/Users/rojcyk/Documents/Comics/Bleach/Volume 004/bleach_031.zip"
echo "${backupPath}/${f#$directoryPath}"
Produces
/Volumes/backMeUp/main//Comics/Bleach/Volume 004/bleach_031.zip
The double slash in the middle is OK. If you don't want it: "${backupPath}/${f#$directoryPath/}"
is this what you want?
kent$ cat t
$foo$bar$blah
kent$ sed 's/\$foo/\$xxx/g' t
$xxx$bar$blah
or this?
kent$ echo $foo
/f/o/o
kent$ echo $bar
/b/a/r
kent$ cat t
something /f/o/o
kent$ sed "s:$foo:$bar:" t
something /b/a/r
sed -e "s/$directoryPath/$backupPath/"
You're on the right path, just need to manipulate the two path variables first.
The contents of $directoryPath is /Users/rojcyk/Documents, so your sed command is coming out as sed -e "s//Users/rojcyk/Documents//Volumes/backMeUp/main/". You need to escape the directory slashes so they don't affect the sed command.
Try this:
directoryPathEscaped=`echo $directoryPath | sed -e "s/\\//\\\\\\\\\//g"`
backupPathEscaped=`echo $backupPath| sed -e "s/\\//\\\\\\\\\//g"`
# At this point, directoryPathEscaped is equal to "\/Users\/rojcyk\/Documents"
# Now you can do your original command line, with the new variables.
sed -e "s/$directoryPathEscaped/$backupPathEscaped/"
Related
It's my 1st script in bash and I'm making a script that the bitbucket pipeline will use. I'm trying to change each 1st letter of the directory to lowerCase.
E.g.
database/Seeders/Entities/Users/Earnings/UserEarningsReportSeeder
will be changed on
database/seeders/entities/users/earnings/userEarningsReportSeeder
Trying with this but not working properly for me :(
echo "$(echo "$Line" | sed 's/\/[A-Z]/\/L&/g')"
Like this (you was very close):
$ sed 's#/[A-Z]#\L&#g' <<< 'Database/Seeders/EarningsFoo'
The separator can be any ASCII character, here #
Output
database/seeders/earningsFoo
I am trying to script a way of removing hosts from the hostgroup file in Nagios Core.
The format of the hostgroup file is:
server1,server2,server3,server4
When removing a server, I need to be able to not only remove the server, but also the comma that follows it. So in my example above, if I am removing server2, the file would result as follows
server1,server3,server4
So I have googled and tested the following which works to remove server2 and a comma after it (I don't know what the b is used for exactly)
sed -i 's/\bserver2\b,//g' myfile
What I want to be able to do is to feed a list of hostnames to a small script to remove a bunch of hosts (and their following comma) with something similar to the following. The problem lies in that placing a variable like $x breaks the script so that nothing happens.
#!/bin/ksh
for x in `cat /tmp/list`
do
sed -i 's/\b${x}\b,//g' myfile
done
I think I am very close on a solution here, but could use a little help. Thanks much in advance for your kind assistance.
Using single quotes tells the shell not to replace the ${x} - it turns off variable interpolation if you want to google for it.
https://www.tldp.org/LDP/abs/html/quotingvar.html. So use double quotes around the sed replacement string instead:
while read -r x; do sed -i "s/\b${x},\b//g" myfile; done < /tmp/list
But since the last field won't have a comma after it, might be a good idea to run two sed commands, one looking for \bword,\b and the other for ,word$ - where \b is a word boundary and $ is the end of line.
while read -r x; do sed -i "s/\b${x},\b//g" myfile; sed -i "s/,${x}$//" myfile ; done < /tmp/list
One other possible boundary condition - what if you have just server2 on a line by itself and that's what you're trying to delete? Perhaps add a third sed, but this one will leave a blank line behind which you might want to remove:
while read -r x
do
sed -i "s/\b${x},\b//g" myfile # find and delete word,
sed -i "s/,${x}$//" myfile # find and delete ,word
sed -i "s/^${x}$//" myfile # find word on a line by itself
done < t
This works quite nicely:
#!/bin/bash
IN_FILE=$1
shift; sed -i "s/\bserver[$#],*\b//g" $IN_FILE; sed -i "s/,$//g" $IN_FILE
if you invoke it like ./remove_server.sh myfile "1 4" for your example file containing server1,server2,server3,server4, you get the following output:
server2,server3
A quick explanation of what it does:
shift shifts the arguments down by one (making sure that "myfile" isn't fed into the regex)
First sed removes the server with the numbers supplied as arguments in the string (e.g. "1 4")
Second sed looks for a trailing comma and removes it
The \b matches a word boundary
This is a great resource for learning about and testing regex: https://regex101.com/r/FxmjO5/1. I would recommend you check it out and use it each time you have a regex problem. It's helped me on so many occasions!
An example of this script working in a more general sense:
I tried it out on this file:
# This is some file containing server info:
# Here are some servers:
server2,server3
# And here are more servers:
server7,server9
with ./remove_server.sh myfile "2 9" and got this:
# This is some file containing info:
# Here are some servers:
server3
# And here are more servers:
server7
Pretty sure there is a pure sed solution for this but here is a script.
#!/usr/bin/env bash
hosts=()
while read -r host; do
hosts+=("s/\b$host,\{,1\}\b//g")
done < /tmp/list
opt=$(IFS=';' ; printf '%s' "${hosts[*]};s/,$//")
sed "$opt" myfile
It does not run sed line-by-line, but only one sed invocation. Just in case, say you have to remove 20+ pattern then sed will not run 20+ times too.
Add the -i if you think the output is ok.
Using perl and regex by setting the servers to a regex group in a shell variable:
$ remove="(server1|server4)"
$ perl -p -e "s/(^|,)$remove(?=(,|$))//g;s/^,//" file
server2,server3
Explained:
remove="(server1|server4)" or "server1" or even "server."
"s/(^|,)$remove(?=(,|$))//g" double-quoted to allow shell vars, remove leading comma, expected to be followed by a comma or the end of string
s/^,// file remove leading comma if the first entry was deleted
Use the -i switch for infile editing.
bash script that reads the servers to remove from standard input, one per line, and uses perl to remove them from the hostfile (Passed as the first argument to the script):
#!/usr/bin/env bash
# Usage: removehost.sh hostgroupfile < listfile
mapfile -t -u 0 servers
IFS="|"
export removals="${servers[*]}"
perl -pi -e 's/,?(?:$ENV{removals})\b//g; s/^,//' "$1"
It reads the servers to remove into an array, joins that into a pipe-separated string, and then uses that in the perl regular expression to remove all the servers in a single pass through the file. Slashes and other funky characters (As long as they're not RE metacharacters) won't mess up the parsing of the perl, because it uses the environment variable instead of embedding the string directly. It also uses a word boundry so that removing server2 won't remove that part of server22.
I am on Mac Os 10.14.6 and have a directory that contains subdirectories that all contain text files. Altogether, there are many hundreds of text files.
I would like to go through the text files and check for any content in the first line that is in parentheses. If such content is found, then the parentheses (and content in the parentheses) should be removed.
Example:
Before removal:
The new world (82 edition)
After removal:
The new world
How would I do this?
Steps I have tried:
Google around, it seems SED would be best for this.
I have found this thread, which provides SED code for removing bracketed content.
sed -e 's/([^()]*)//g'
However, I am not sure how to adapt it to work on multiple files and also to limit it to the first line of those files. I found this thread which explains how to use SED on multiple files, but I am not sure how to adapt the example to work with parentheses content.
Please note: As long as the solution works on Mac OS terminal, then it does not need to use SED. However, from Googling, SED seems to be the most suited.
I managed to achieve what you're after simply by using a bash script and sed together, as so:
#!/bin/bash
for filename in $PWD/*.txt; do
sed -i '' '1 s/([^()]*)//g' $filename
done
The script simply iterates over all the .txt files in $PWD (the current working directory, so that you can add this script to your bin and run it anywhere), and then runs the command
sed -ie '1 s/([^()]*)//g' $filename
on the file. By starting the command with the number 1 we tell sed to only work on the first line of the file :)
Edit: Best Answer
The above works fine in a directory where all contained objects are files, and not including directories; in other words, the above does not perform recursive search through directories.
Therefore, after some research, this command should perform exactly what the question asks:
find . -name "*.txt" -exec sed -i '' '1 s/([^()]*)//g' {} \;
I must iterate, and reiterate, that you test this on a backup first to test it works. Otherwise, use the same command as above but change the '' in order to control the creation of backups. For example,
find . -name "*.txt" -exec sed -i '.bkp' '1 s/([^()]*)//g' {} \;
This command will perform the sed replace in the original file (keeping the filename) but will create a backup file for each with the appended .bkp, for example test1.txt becomes test1.txt.bkp. This a safer option, but choose what works best for you :)
Good try,
The command you where looking for single line:
sed -E '1s|\([^\)]+\)||'
The command to replace each input file first line:
sed -Ei '1s|\([^\)]+\)||' *.txt
example:
echo "The new world (82 edition)" |sed -E '1s|\([^\)]+\)||'
The new world
Explanation
sed -Ei E option: the extended RegExp syntax, i option: for in-place file replacement
sed -Ei '1s|match RegExp||' for first line only, replace first matched RegExp string with empty string
\([^\)]+\) RegExp matching: start with (, [^\)]any char not ), + - more than once, terminate with )
Try:
# create a temporary file
tmp=$(mktemp)
# for each something in _the current directory_
for i in *; do
# if it is not a file, don't parse it
if [ ! -f "$i" ]; then continue; fi
# remove parenthesis on first line, save the output in temporary file
sed '1s/([^)]*)//g' "$i" > "$tmp"
# move temporary file to the original file
mv "$tmp" "$i"
done
# remove temporary file
rm "$tmp"
I want a command the will assing the name of the 2 file/folder in a folder to a variable. No files/folders are specific, I want this command to work on any folder.
For example:
If a do an ls in a folder it gives me:
hey.txt
hello
lol
yolo.png
And I want a command finding the second file/folder, in this case it is the folder hello. Then i want $WATHFILE to have the value hello.
Thanks in advance.
Edit: Also, is there a way to change the what number of file to find, from the second file to the third, based on a variable?
This will set a variable with the second result:
var=$(ls | sed -n '2p')
Why do you want to set the builtin variable $8 to this value?
This sounds like a job for sed the stream editor. You can pipe the output of ls through sed then extract the second word:
ls | sed -n 2p
Now you can do what ever you want with this output, such as store it to a variable:
foo=`ls | sed -n 2p`
I have different location, but they all have a pattern:
some_text/some_text/some_text/log/some_text.text
All locations don't start with the same thing, and they don't have the same number of subdirectories, but I am interested in what comes after log/ only. I would like to extract the .text
edited question:
I have a lot of location:
/s/h/r/t/log/b.p
/t/j/u/f/e/log/k.h
/f/j/a/w/g/h/log/m.l
Just to show you that I don't know what they are, the user enters these location, so I have no idea what the user enters. The only I know is that it always contains log/ followed by the name of the file.
I would like to extract the type of the file, whatever string comes after the dot
THe only i know is that it always contains log/ followed by the name
of the file.
I would like to extract the type of the file, whatever string comes
after the dot
based on this requirement, this line works:
grep -o '[^.]*$' file
for your example, it outputs:
text
You can use bash built-in string operations. The example below will extract everything after the last dot from the input string.
$ var="some_text/some_text/some_text/log/some_text.text"
$ echo "${var##*.}"
text
Alternatively, use sed:
$ sed 's/.*\.//' <<< "$var"
text
Not the cleanest way, but this will work
sed -e "s/.*log\///" | sed -e "s/\..*//"
This is the sed patterns for it anyway, not sure if you have that string in a variable, or if you're reading from a file etc.
You could also grab that text and store in a sed register for later substitution etc. All depends on exactly what you are trying to do.
Using awk
awk -F'.' '{print $NF}' file
Using sed
sed 's/.*\.//' file
Running from the root of this structure:
/s/h/r/t/log/b.p
/t/j/u/f/e/log/k.h
/f/j/a/w/g/h/log/m.l
This seems to work, you can skip the echo command if you really just want the file types with no record of where they came from.
$ for DIR in *; do
> echo -n "$DIR "
> find $DIR -path "*/log/*" -exec basename {} \; | sed 's/.*\.//'
> done
f l
s p
t h