I have a text file that has a list of filenames.Now I want to see if these files are present in a specific folder using bash script. I have no experience in writing bash scripts.So please help me with this
a simple for loop can do what you need :
for file in $(cat yourfilelist.txt); do ls -l /folder/to/check/"$file"; done
for missing files you'd get No such file or directory
Related
I like to create tar-files to distribute some scripts using bash.
For every script certain configuration-files and libraries (or toolboxes) are needed,
e.g. a script called CheckTool.py needs Checks.ini, CheckToolbox.py and CommontToolbox.py to run, which are stored in specific folders on my harddisk and need to be copied in the same manner on the users harddisk.
I can create a tarfile manually for each script, but i like to have it more simple.
For this i have the idea to define a list of all needed files and their pathes for a specific script and read this in a bashscript, which creates the tar file.
I started with:
#!/bin/bash
while read line
do
echo "$line"
done < $1
Which is reading the files and pathes. In my example the lines are:
./CheckTools/CheckMesh.bs
./Configs/CheckMesh.ini
./Toolboxes/CommonToolbox.bs
./Toolboxes/CheckToolbox.bs
My question is how do I have to organize the data to make a tar file with the specified files using bash?
Or is there someone having a better idea?
No need for a complicated script, use option -T of tar. Every file listed in there will be added to the tar file:
-T, --files-from FILE
get names to extract or create from FILE
So your script becomes:
#!/bin/bash
tar -cvpf something.tar -T listoffiles.txt
listoffiles.txt format is super easy, one file per line. You might want to put full path to ensure you get the right files:
./CheckTools/CheckMesh.bs
./Configs/CheckMesh.ini
./Toolboxes/CommonToolbox.bs
./Toolboxes/CheckToolbox.bs
You can add tar commands to the script as needed, or you could loop on the list files, from that point on, your imagination is the limit!
Basically I have .bz2.gz.bz2 file which on extraction gives a .bz2.gz file and on again extraction gives .bz2 file. In this .bz2 file, is my txt file which I want to search on using grep command. I have searched for this but I got bzgrep command which will only search in bz2 file and not the corresponding .gz.bz2 file and give me no results.
Is there a command in unix system which will recursively search in a zipped archive for zipped archive and return results only when it finds the txt file inside it?
P.S: the txt file may be deep in the archive to level 10 max. I want the command to recursively find the txt file and search for the required string. And there will be no other than an archive inside the archive until the txt file level.
I'm not sure I fully understand but maybe this will help:
for i in /path/to/each/*.tar.bz2; do
tar -xvjf "$i" -C /path/to/save/in
rm $i
done
extract all `tar.bz2` and save them in directory then remove the `.bz2`
Thnx for sharing your question.
There are a couple of strange things with it though:
It makes no sense to have a .bz2.gz.bz2 file, so have you created this file yourself? If so, I'd advise you to reconsider doing so in that manner.
Also, you mention there is a .bz2 that would apparently contain different archives, but a .bz2 can only contain one single file by design. So if it contains archives it is probably a .tar.bz2 file in which the tar-file holds the actual archives.
In answer to your question, why can't you write a simple shell script that will unpack your .bz2.gz.bz2 into a .bz2.gz and then into a .bz2 file and then execute your bzgrep command on that file?
I do not understand where it is exactly that you seem to get stuck..
I'm currently running a Bash script called (log2csv) that runs against a specified .log file. Sitting in the desired directory I can type in terminal:
log2csv Red1_1.log
This will create Red1_1.csv
This is my current bash script:
#!/bin/bash
for path
do
base=$(basename "$path")
noext="${base/.log}"
/Users/joshuacarter/bin/read_scalepack.pl "$path" > "${noext}.csv"
done
This script is actually running a perl script on the specified log and putting the results in a CSV output.
I can alternatively run in terminal:
log2csv *.log
This will run the script against all .log files in the current directory and create .csv files for every one.
What I would like the script to do is only run on .log files that haven't had .csv files created for them. After doing some research I think I possibly can use inotifywait to achieve this, but I'm unsure how to make this work in my script? I also have read that this may be an issue if you overwrite a file. Any help or ideas would be most appreciated!
What I would like the script to do is only run on .log files that haven't had .csv files created for them.
Simply skip those .log files whose corresponding .csv files already exist:
for path
do
base=$(basename "$path")
noext="${base/.log}"
[ -e "${noext}.csv" ] && continue # <---------------
/Users/joshuacarter/bin/read_scalepack.pl "$path" > "${noext}.csv"
done
I have a folder in which there are many many folder and in each of these I have lots and lots of files. I have no idea which folder each files might be located in. I will periodically receive a list of files I need to copy to a predefined destination.
The script will run on a Unix machine.
So, my little script should:
read received list
find all files in the list
copy each file to a predefined destination via SCP
step 1 and 3, I think I'll manage on my own, but how will I do step 2?
I was thinking about using "find" to locate each file and when found, write the location in a string array. When all files are found I loop through the string array, running the "SCP" command for each file-location.
I think this should work, but I've never written a bash script before so could anyone help me a little to get started? I just need a basic "find" command which finds a filename and returns the file location if the file is found.
find $dir -name $name -exec scp {} $destination \;
To create a playlist for all of the music in a folder, I am using the following command in bash:
ls > list.txt
I would like to use the result of the pwd command for the name of the playlist.
Something like:
ls > ${pwd}.txt
That doesn't work though - can anyone tell me what syntax I need to use to do something like this?
Edit: As mentioned in the comments pwd will end up giving an absolute path, so my playlist will end up being named .txt in some directory - d'oh! So I'll have to trim the path. Thanks for spotting that - I would probably have spent ages wondering where my files went!
The best way to do this is with "$(command substitution)" (thanks, Landon):
ls > "$(pwd).txt"
You will sometimes also see people use the older backtick notation, but this has several drawbacks in terms of nesting and escaping:
ls > "`pwd`.txt"
Note that the unprocessed substitution of pwd is an absolute path, so the above command creates a file with the same name in the same directory as the working directory, but with a .txt extension. Thomas Kammeyer pointed out that the basename command strips the leading directory, so this would create a text file in the current directory with the name of that directory:
ls > "$(basename "$(pwd)").txt"
Also thanks to erichui for bringing up the problem of spaces in the path.
This is equivalent to the backtick solution:
ls > $(pwd).txt
To do literally what you said, you could try:
ls > `pwd`.txt
which will use the full pathname, which should be fine.
Note that if you do this in your home directory, which might
be in /home/hoboben, you will be trying the create /home/hoboben.txt,
a text file in the directory above.
Is this what you wanted?
If you wanted the directory to contain a file named after it, you would get
the basename of the current directory and append that with .txt to the pwd.
Now, rather than use the pwd command... why not use the PWD environment variable?
For example:
ls > $PWD.txt
or
ls > ${PWD}.txt
is probably what you were trying to remember with your second example.
If you're in /home/hoboben and you want to create /home/hoboben/hoboben.txt, try:
ls > ${PWD}/${PWD##*/}.txt
If you do this, the file will contain its own name, so most often, you would remedy this in one of a few ways. You could redirect to somewhere else and move the file or name the file beginning with a dot to hide it from the ls command as long as you don't use the -a flag (and then optionally rename the resulting file).
I write my own scripts to manage a directory hierarchy of music files and I use subdirectories named ".info", for example, to contain track data in some spare files (basically, I "hide" metadata this way). It works out okay because my needs are simple and my collection small.
I suspect the problem may be that there are spaces in one of the directory names. For example, if your working directory is "/home/user/music/artist name". Bash will be confused thinking that you are trying to redirect to /home/user/music/artist and name.txt. You can fix this with double quotes
ls > "$(pwd).txt"
Also, you may not want to redirect to $(pwd).txt. In the example above, you would be redirecting the output to the file "/home/user/music/artist name.txt"
The syntax is:
ls > `pwd`.txt
That is the '`' character up underneath the '~', not the regular single quote.
Using the above method will create the files one level above your current directory. If you want the play lists to all go to one directory you'd need to do something like:
#!/bin/sh
MYVAR=`pwd | sed "s|/|_|g"`
ls > /playlistdir/$MYVAR-list.txt
to strip all but the directory name
ls >/playlistdir/${PWD##/*}.txt
this is probably not what you want because then you don't know where the files are (unless you change the ls command)
to replace "/" with "_"
ls >/playlistdir/${PWD//\//_}.txt
but then the playlist would look ugly and maybe not even fit in the selection window
So this will give you both a short readable name and usable paths inside the file
ext=.mp3 #leave blank for all files
for FILE in "$PWD/*$ext"; do echo "$FILE";done >/playlistdir/${PWD##/*}.txt