I am trying to create a version of file copy that allows the user to append the target file that already exists. The program takes two arguments: source-file and destination-file.
If the destination-file already exists, it must ask the user to choose between three options: overwrite destination-file, append destination-file or cancel.
It should make the following argument checks:
The number of arguments must be 2.
The source-file must be a regular file.
If the destination-file exists, it must be a regular file.
Can anyone help me how I would go about completing this task?
An attempt to answer your 4 or 5 questions: In the future, please ask a single question per post and make an attempt at answering it as all of this is pretty easily answered through google.
How to check how many arguments are passed to the script
Information on "test". You can also use command man test to RTFM
Some examples:
Testing if it's a regular file (from the previous link) returns true/false:
[ -f myfile ]
Testing if file exists (again from previous link) returns true/false:
[ -a myfile ]
How to read user input
I think the "inline" bash version here is very nice for your needs:
read -p "Do you want to (C)opy, (A)ppend, or (E)nd? " answer
case ${answer:0:1} in
c|C )
cp file1 file2
;;
a|A )
cat file1 >> file2
;;
* )
echo "That's not an option, yo"
;;
esac
How to copy a file (after user makes selection)
cp file1 file2
How to append a file (after user makes selection)
cat file >> file2
Related
Hello I am trying to get all files with Jane's name to a separate file called oldFiles.txt. In a directory called "data" I am reading from a list of file names from a file called list.txt, from which I put all the file names containing the name Jane into the files variable. Then I'm trying to test the files variable with the files in list.txt to ensure they are in the file system, then append the all the files containing jane to the oldFiles.txt file(which will be in the scripts directory), after it tests to make sure the item within the files variable passes.
#!/bin/bash
> oldFiles.txt
files= grep " jane " ../data/list.txt | cut -d' ' -f 3
if test -e ~data/$files; then
for file in $files; do
if test -e ~/scripts/$file; then
echo $file>> oldFiles.txt
else
echo "no files"
fi
done
fi
The above code gets the desired files and displays them correctly, as well as creates the oldFiles.txt file, but when I open the file after running the script I find that nothing was appended to the file. I tried changing the file assignment to a pointer instead files= grep " jane " ../data/list.txt | cut -d' ' -f 3 ---> files=$(grep " jane " ../data/list.txt) to see if that would help by just capturing raw data to write to file, but then the error comes up "too many arguments on line 5" which is the 1st if test statement. The only way I get the script to work semi-properly is when I do ./findJane.sh > oldFiles.txt on the shell command line, which is me essentially manually creating the file. How would I go about this so that I create oldFiles.txt and append to the oldFiles.txt all within the script?
The biggest problem you have is matching names like "jane" or "Jane's", etc. while not matching "Janes". grep provides the options -i (case insensitive match) and -w (whole-word match) which can tailor your search to what you appear to want without having to use the kludge (" jane ") of appending spaces before an after your search term. (to properly do that you would use [[:space:]]jane[[:space:]])
You also have the problem of what is your "script dir" if you call your script from a directory other than the one containing your script, such as calling your script from your $HOME directory with bash script/findJane.sh. In that case your script will attempt to append to $HOME/oldFiles.txt. The positional parameter $0 always contains the full pathname to the current script being run, so you can capture the script directory no matter where you call the script from with:
dirname "$0"
You are using bash, so store all the filenames resulting from your grep command in an array, not some general variable (especially since your use of " jane " suggests that your filenames contain whitespace)
You can make your script much more flexible if you take the information of your input file (e.g list.txt), the term to search for (e.g. "jane"), the location where to check for existence of the files (e.g. $HOME/data) and the output filename to append the names to (e.g. "oldFile.txt") as command line [positonal] parameters. You can give each default values so it behaves as you currently desire without providing any arguments.
Even with the additional scripting flexibility of taking the command line arguments, the script actually has fewer lines simply filling an array using mapfile (synonymous with readarray) and then looping over the contents of the array. You also avoid the additional subshell for dirname with a simple parameter expansion and test whether the path component is empty -- to replace with '.', up to you.
If I've understood your goal correctly, you can put all the pieces together with:
#!/bin/bash
# positional parameters
src="${1:-../data/list.txt}" # 1st param - input (default: ../data/list.txt)
term="${2:-jane}" # 2nd param - search term (default: jane)
data="${3:-$HOME/data}" # 3rd param - file location (defaut: ../data)
outfn="${4:-oldFiles.txt}" # 4th param - output (default: oldFiles.txt)
# save the path to the current script in script
script="$(dirname "$0")"
# if outfn not given, prepend path to script to outfn to output
# in script directory (if script called from elsewhere)
[ -z "$4" ] && outfn="$script/$outfn"
# split names w/term into array
# using the -iw option for case-insensitive whole-word match
mapfile -t files < <(grep -iw "$term" "$src" | cut -d' ' -f 3)
# loop over files array
for ((i=0; i<${#files[#]}; i++)); do
# test existence of file in data directory, redirect name to outfn
[ -e "$data/${files[i]}" ] && printf "%s\n" "${files[i]}" >> "$outfn"
done
(note: test expression and [ expression ] are synonymous, use what you like, though you may find [ expression ] a bit more readable)
(further note: "Janes" being plural is not considered the same as the singular -- adjust the grep expression as desired)
Example Use/Output
As was pointed out in the comment, without a sample of your input file, we cannot provide an exact test to confirm your desired behavior.
Let me know if you have questions.
As far as I can tell, this is what you're going for. This is totally a community effort based on the comments, catching your bugs. Obviously credit to Mark and Jetchisel for finding most of the issues. Notable changes:
Fixed $files to use command substitution
Fixed path to data/$file, assuming you have a directory at ~/data full of files
Fixed the test to not test for a string of files, but just the single file (also using -f to make sure it's a regular file)
Using double brackets — you could also use double quotes instead, but you explicitly have a Bash shebang so there's no harm in using Bash syntax
Adding a second message about not matching files, because there are two possible cases there; you may need to adapt depending on the output you're looking for
Removed the initial empty redirection — if you need to ensure that the file is clear before the rest of the script, then it should be added back, but if not, it's not doing any useful work
Changed the shebang to make sure you're using the user's preferred Bash, and added set -e because you should always add set -e
#!/usr/bin/env bash
set -e
files=$(grep " jane " ../data/list.txt | cut -d' ' -f 3)
for file in $files; do
if [[ -f $HOME/data/$file ]]; then
if [[ -f $HOME/scripts/$file ]]; then
echo "$file" >> oldFiles.txt
else
echo "no matching file"
fi
else
echo "no files"
fi
done
I am trying to write a script that uses agrep to loop through files in one document and match them against another document. I believe this might use a nested loop however, I am not completely sure. In the template document, I need for it to take one string and match it against other strings in another document then move to the next string and match it again
If unable to see images for some odd reason I have included the links at the bottom here as well. Also If you need me to explain more just let me know. This is my first post so I am not sure how this will be perceived or if I used the correct terminologies :)
Template agrep/highlighted- https://imgur.com/kJvySbW
Matching strings not highlighted- https://imgur.com/NHBlB2R
I have already looked on various websites regarding loops
#!/bin/bash
#agrep script
echo ${BASH_VERSION}
TemplateSpacers="/Users/kj/Documents/Research/Dr. Gage
Research/Thesis/FastA files for AGREP
test/Template/TA21_spacers.fasta"
MatchingSpacers="/Users/kj/Documents/Research/Dr. Gage
Research/Thesis/FastA files for AGREP test/Matching/TA26_spacers.fasta"
for * in filename
do
agrep -3 * to file im comparing to
#potentially may need to use nested loop but not sure
Ok, I get it now, I think. This should get you started.
#!/bin/bash
document="documentToSearchIn.txt"
grep -v spacer fileWithSearchStrings.txt | while read srchstr ; do
echo "Searching for $srchstr in $document"
echo agrep -3 "$srchstr" "$document"
done
If that looks correct, remove the echo before agrep and run again.
If, as you say in the comments, you want to store the script somewhere else, say in $HOME/bin, you can do this:
mkdir $HOME/bin
Save the script above as $HOME/bin/search. Now make it executable (only necessary one time) with:
chmod +x $HOME/bin/search
Now add $HOME/bin to your PATH. So, find the line starting:
export PATH=...
in your login profile, and change it to include the new directory:
export PATH=$PATH:$HOME/bin
Then start a new Terminal and you should be able to just run:
search
If you want to be able to specify the name of the strings file and the document to search in, you can change the code to this:
#!/bin/bash
# Pick up parameters, if supplied
# 1st param is name of file with strings to search for
# 2nd param is name of document to search in
str=${1:-""}
doc=${2:-""}
# Ensure name of strings file is valid
while : ; do
[ -f "$str" ] && break
read -p "Enter strings filename:" str
done
# Ensure name of document file is valid
while : ; do
[ -f "$doc" ] && break
read -p "Enter document name:" doc
done
echo "Search for strings from: $str, searching in document: $doc"
grep -v spacer "$str" | while read srchstr ; do
echo "Searching for $str in $doc"
echo agrep -3 "$str" "$doc"
done
Then you can run:
search path/to/file/with/strings path/to/document/to/search/in
or, if you run like this:
search
it will ask you for the 2 filenames.
I have some pseduocode below and would like to know if it would work/ is the best method to tackle the problem before I begin developing the code.
I need to dynamically search through a directory on one server and find out if it exists on another server or not. The path will be different so I use basename and save it as a temporary variable.
for $FILE in $CURRENT_DIRECTORY
$TEMP=$(basename "$FILE" )
if [ssh user#other_serverip find . -name '$TEMP']; then
//write code here
fi
Would this if statement return true if the file existed on the other server?
Here is a functioning, cleaner implementation of your logic:
for FILE in *; do
if ssh user#other_serverip test -e "$FILE"; then
# write code here
fi
done
(There won't be a path on files when the code is composed this way, so you don't need basename.) test -e "$FILE" will silently exit 0 (true) if the file exists and 1 (false) if the file does not, though ssh will also exit with a false code if the connection fails.
However, that is a very expensive way to solve your issue. It will fail if your current directory has too many files in it and it runs ssh once per file.
You're better off getting a list of the remote files first and then checking against it:
#!/bin/sh
if [ "$1" != "--xargs" ]; then # this is an internal flag
(
ssh user#other_serverip find . -maxdepth 1 # remote file list
find . -maxdepth 1 # local file list
) |awk '++seen[$0]==2' |xargs -d "\n" sh "$0" --xargs # keep duplicates
else
shift # remove the --xargs marker
for FILE in "$#"; do
# write code here using "$FILE" (with quotes)
done
fi
This does two things. First, since the internal --xargs is not given when you run the script, it connects to the remote server and gets a list of all files in the home directory there. These will be listed as ./.bashrc for example. Then the same list is generated locally, and the results are passed to awk.
The awk command builds an associative array (a hash) from each item it sees, incrementing it and then checking the total against the number two. It prints the second instance of any line it sees. Those are then passed on to xargs, which is instructed to use \n (a line break) as its delimiter rather than any space character.
Note: this code will break if you have any files that have a line break in their name. Don't do that.
xargs then recursively calls this script, resulting in the else clause and we loop through each file. If you have too many files, be aware that there may be more than one instance of this script (see man xargs).
This code requires GNU xargs. If you're on BSD or some other system that doesn't support xargs -d "\n", you can use perl -pe 's/\n/\0/' |xargs -0 instead.
It would return true if ssh exits successfully.
Have you tried command substitution and parsing find's output instead?
I have made a directory with lots of files with:
samplefile_111222015_reporting_{1..13}
I am trying to create a vi script where when I enter the directory as an argument to the command e.g.
sh myScript $HOME/theDir/*
then it copies all the files in that directory to a new one I made. Although right now, I'm having problems with the for loop alone.
This is what I have in my script:
for f in $1;
do
echo "$f"
done
but when i enter sh myScript $HOME/theDir, I get back the name of the first file only (samplefile_111222015_reporting_1). why the first file? Is this not a for loop>
# Because of the wild card expansion, all the files in the directory are
# already made available to the script through arguments
# So do the following to get all the file listing
for f ; do echo $f; done
This is because each file is passed as a separate argument and you're only looping over $1, which is the first argument.
Instead, you most likely want to loop over "$#", which is every argument starting from $1.
The man page for bash, under the Special Parameters section, details the special parameters available in more detail.
I would like to get just the filename (with extension) of the output file I pass to my bash script:
a=$1
b=$(basename -- "$a")
echo $b #for debug
if [ "$b" == "test" ]; then
echo $b
fi
If i type in:
./test.sh /home/oscarchase/test.sh > /home/oscarchase/test.txt
I would like to get:
test.txt
in my output file but I get:
test.sh
How can I procede to parse this first argument to get the right name ?
Try this:
#!/bin/bash
output=$(readlink /proc/$$/fd/1)
echo "output is performed to \"$output\""
but please remember that this solution is system-dependent (particularly for Linux). I'm not sure that /proc filesystem has the same structure in e.g. FreeBSD and certainly this script won't work in bash for Windows.
Ahha, FreeBSD obsoleted procfs a while ago and now has a different facility called procstat. You may get an idea on how to extract the information you need from the following screenshot. I guess some awk-ing is required :)
Finding out the name of the file that is opened on file descriptor 1 (standard output) is not something you can do directly in bash; it depends on what operating system you are using. You can use lsof and awk to do this; it doesn't rely on the proc file system, and although the exact call may vary, this command worked for both Linux and Mac OS X, so it is at least somewhat portable.
output=$( lsof -p $$ -a -d 1 -F n | awk '/^n/ {print substr($1, 2)}' )
Some explanation:
-p $$ selects open files for the current process
-d 1 selects only file descriptor 1
-a is use to require both -p and -d apply (the default is to show all files that match either condition
-F n modifies the output so that you get one line per field, prefixed with an identifier character. With this, you'll get two lines: one beginning with p and indicating the process ID, and one beginning with `n indicating the file name of the file.
The awk command simply selects the line starting with n and outputs the first field minus the initial n.