I have a script that takes as an argument a path to a file upon which it performs certain operations. These files are stored in directories with path storage///_id/files (so in 2016 July 22 it would be storage/2016/Jul/22_1/files for the first set of files, .../Jul/22_2/files for second one etc.). The problem is each directory stores files with two extensions (say file.doc, file.txt) and I want to perform operations only on .txt files. I've tested earlier something like
for file in "/home/gonczor/temp/"*/*".txt"; do
echo "$file"
done
And it worked perfectly given that names in directories don't change. When I move one step further and add this 22_1, 22_2, 23_1 directories something strange happens.
This is my script (simplified):
for file in "$FILE_PATH/""$YEAR/""$MONTH/""$DAY"*/*".txt"; do
my_program ${report}
done
And instead of finding .../2016/Jul/22_1/file.txt it finds /2016/Jul/22*/*.txt
How can I make it work? The solution I've tried to make up is from here
Related
I need to combine all the csv files in some directory (.csv), provided that there are other files with the same name in this directory, but with different expansion (.csv.done).
If a csv file doesn't have .done in this extension then I don't need it for combine process.
What is the best way to do it using Bash ?
This approach is a solution to your problem. I see you've commented that it "didn't work", but whatever the reason is for it not working, it's likely simple to fix e.g. if you forgot to include key details, or failed to adapt it appropriately to suit your specific situation. If you need further help troubleshooting, add more info to your question.
The approach:
for f in *.csv.done
do
cat "${f%.*}" >> combined_file.csv
done
How it works:
In your example, you have 3 files named 1.csv 2.csv 3.csv and two 'done' files named 1.csv.done 2.csv.done.
This script begins by making a list of all files that end in .csv.done (two files: 1.csv.done 2.csv.done).
It then uses a parameter expansion, specifically ${parameter%word}, to 'shorten' the name of the two files in the list to .csv (instead of .csv.done).
Then it 'prints' the content of the two 'shortened' filenames (1.csv and 2.csv) into a 'combined' file.
It doesn't 'print' the content of 1.csv.done or 2.csv.done, or 3.csv, because these files weren't in the original 'list'.
If you run this script multiple times, it will keep adding the contents of files 1.csv and 2.csv to the 'combined' file (only run it once, or delete the 'combined' file before running it again)
I need to make shell script to check my algorithms with loads of data(tests packages saved in .in files, every package contains folder with .in file and the other one with .out file where supposed to be correct result)
Sometimes It's about 1000 files in one packages so there's no point of doing it manually. I need some kind of loop which opens this .in file then redirect input of my c++ program and also redirect output of this program(save result to .out files) But the point is I can't get this language as quick as I need.
And I would like this script to compare results of my algorithm to .out files from packages
for f in ExternalIn/*.in; do//part of code which opens process with my algorithm and compare its .out file to .out file from package
Skipping checks for missing files, whitespace-safety, etc., you probably need something like:
for f in ExternalIn/*.in; do
# diff the result of my_cpp_app eating file.in with file.out
# and store the comparison result in file.diff
diff ${f/.in/.out} <(my_cpp_app <$f 2>/dev/null) > ${f/.in/.diff}
done
Although I would probably do it with find / xargs pipeline which is not only safer but also allows parallel execution.
Or even write a Makefile for this and use make, which after all is a tool for exactly this kind of work.
I have a bunch of files I'm trying to organize quickly, and I had two questions about how to do that. I really appreciate any help! I tried searching but couldn't find anything on these specific commands for OSX.
First, I have about 100 folders in a directory - I'd like to place an folder in each one of those folders.
For example, I have
Cars/Mercedes/<br>
Cars/BMW/<br>
Cars/Audi/<br>
Cars/Jeep/<br>
Cars/Tesla/
Is there a way I can create a folder inside each of those named "Pricing" in one command, i.e. ->
Cars/Mercedes/Pricing <br>
Cars/BMW/Pricing<br>
Cars/Audi/Pricing<br>
Cars/Jeep/Pricing<br>
Cars/Tesla/Pricing
My second question is a little tougher to explain. In each of these folders, I'd like move certain files into these newly created folders (above) in the subdirectory.
Each file has a slightly different filename but contains the same string of letters - for example, in each of the above folders, I might have
Cars/Mercedes/payment123.html
Cars/BMW/payment432.html
Cars/Audi/payment999.html
Cars/Jeep/payment283.html
Is there a way to search each subdirectory for a file containing the string "payment" and move that file into a subfolder in that subdirecotry - i.e. into the hypothetical "Pricing" folders we just created above with one command for all the subdirectories in Cars?
Thanks so much~! help with either of these would be invaluable.
I will assume you are using bash, since it is the default shell in OS X. One way to do this uses a for loop over each directory to create the subdirectory and move the file. Wildcards are used to find all of the directories and the file.
for DIR in Cars/*/ ; do
mkdir "${DIR}Pricing"
mv "${DIR}payment*.html" "${DIR}Pricing/"
done
The first line finds every directory in Cars, and then runs the loop once for each, replacing ${DIR} with the current directory. The second line creates the subdirectory using the substitution. Note the double quotes, which are necessary only if the path could contain spaces. The third line moves any file in the directory whose name starts with "payment" and ends with ".html" to the subdirectory. If you have multiple files which match this, they will all be moved. The fourth line simply marks the end of the loop.
If you are typing this directly into the command line, you can combine it into a single line:
for DIR in Cars/*/ ; do mkdir "${DIR}Pricing"; mv "${DIR}payment*.html" "${DIR}Pricing/"; done
I have to use bash scripting to copy files from one folder to another. If the destination folder has a file with the same name but older timestamp, it should not copy. Only newer files should be copied. I could have used cp -u, but I was asked not to use it. Essentially I have to use the test command testing for "ot". Please let me know how could this be done. I believe two for loops one to read the files in the source and one for the destination directories can be used and the the time stamp compared. The problem is that both for loops produce the absolute path names along with the file name. So not sure how to compare them
Thanks
You can profit from the parameter substitution:
for file in "$folder1"/* ; do
filename=${file##*/} # Remove everything to the last slash.
Or, you can change the directory:
cd "$folder1"
for file in * ; do
## you have to use full or relative path to $folder2 here
My websites file structure has gotten very messy over the years from uploading random files to test different things out. I have a list of all my files such as this:
file1.html
another.html
otherstuff.php
cool.jpg
whatsthisdo.js
hmmmm.js
Is there any way I can input my list of files via command line and search the contents of all the other files on my website and output a list of the files that aren't mentioned anywhere on my other files?
For example, if cool.jpg and hmmmm.js weren't mentioned in any of my other files then it could output them in a list like this:
cool.jpg
hmmmm.js
And then any of those other files mentioned above aren't listed because they are mentioned somewhere in another file. Note: I don't want it to just automatically delete the unused files, I'll do that manually.
Also, of course I have multiple folders so it will need to search recursively from my current location and output all the unused (unreferenced) files.
I'm thinking command line would be the fastest/easiest way, unless someone knows of another. Thanks in advance for any help that you guys can be!
Yep! This is pretty easy to do with grep. In this case, you would run a command like:
$ for orphan in `cat orphans.txt`; do \
echo "Checking for presence of ${orphan} in present directory..." ;
grep -rl $orphan . ; done
And orphans.txt would look like your list of files above, one file per line. You can add -i to the grep above if you want to grep case-insensitively. And you would want to run that command in /var/www or wherever your distribution keeps its webroots. If, after you see the above "Checking for..." and no matches below, you haven't got any files matching that name.