I have a cloned git repo that I'm working with. In each folder is a file called config.js. I have a script that SHOULD convert each file in the repo, ending in .js, to a csv and rename it to foldername.config.js.csv. What Bash loop should I use to accomplish this? Forgive me, for I don't really know Bash at all, but from what I was able to figure out I did:
for NAME in `find . -iname 'config.js'`; do
DIRNAME=`echo $NAME | cut -d \/ -f 3`
./do_conversion_script.sh $NAME
mv config.js.csv $DIRNAME.config.js.csv
done
This successfully applied the script to all the files but it only converted the file and added .csv to the end of it. So instead of giving me foldername.config.js.csv, it just gave me config.js.csv. Any help is appreciated. Thank you.
At first let me make some assumptions:
Your directory tree looks like:
./dir/a/config.js
./dir/b/config.js
etc.
The do_conversion_script.sh creates the config.js.csv file in the
same directory as config.js.
You want to prepend a suffix to the config.js.csv file with
its parent name as a or b in case of the example above.
The renamed file is located in the same directory as before.
The parent names do not duplicate.
The pathnames do not include any of whitespace, tab and newline.
Then how about:
for name in $(find . -iname 'config.js'); do
dir="${name%/*}"
prefix="${dir##*/}"
./do_conversion_script.sh "$name"
mv -- "$dir/config.js.csv" "$dir/$prefix.config.js.csv"
done
You may need to use two variables which holds the full directory name from the current directory and the one which holds just the parent name.
Hope this helps.
Your way of calculating DIRNAME does not make sense. Say if I have
NAME=a/b/c
echo $NAME | cut -d \/ -f 3
this produces c, not a/b.
You could use
DIRNAME=$(dirname $NAME)
or
DIRNAME=${NAME%/*}
instead. The former is better readable, the latter saves ones child process.
I would like to append the same string for all of the folders name (to the end) in the root folder.
Based on the below question these code may be useful:
for dir in * ; do mv "${dir}" "${dir/_-_ppfsefs/}" ; done
Batch rename folders
However, these code cuts the unwanted parts. And I want to add a string at the end of folder name.
use mv "${dir}" "${dir}_-_ppfsefs", no slashes.
I am new to shell scripting.I have a requirement, find out or list .bak(it shld be only .bak, shouldn't select other .bak.tar.gz files while listing) files and then compress them and create .tar.gz files.
Hope these code helps. *.bak will match all .bak files.
tar -czf example.tar.gz *.bak
You probably needs this :
#uncomment 1 of the next 2 lines : the first DOES the cmd, the 2nd only echo it
#thecmd="tar"
thecmd="echo tar"
export thecmd
for bakfile in ./*.bak ; do
[ -f "${bakfile}" ] && ${thecmd} -czf "${bakfile}.tar.gz" "${bakfile}"
done
This also test if the file is a REGULAR file (and not a dir, or a pipe, or a block device) before compressing it into file.tar.gz (note: try to avoid reserved words and variable names in your variable names. There are quite a lot)
I'm making a shell script that I need to make a loop. I have a directory called Files. Inside Files there are two folders. Each holding 500 files (folder1 and folder2). I need to get the filenames from both folder1 and folder2 because I need to concatenate the filename in folder1 with folder2. It needs to do this for every single file in there. So 250,000 times.
Anyone know how I would write that loop so that I can get all the names from both directories and loop correctly?
Assuming you're in bash, then something like this
cd Files
for f1 in folder1/*
do
for f2 in folder2/*
do
concat_name="${f1#*/}-${f2#*/}"
done
done
something like this should do, assuming that the two subdirectories are called dir1 and dir2, this example only echoes the names naturally..
#!/bin/bash
for d1 in `ls Files/dir1`;
do
for d2 in `ls Files/dir2`;
do
echo ${d1}_${d2}
done
done
It's been a while since I've used make, so bear with me...
I've got a directory, flac, containing .FLAC files. I've got a corresponding directory, mp3 containing MP3 files. If a FLAC file is newer than the corresponding MP3 file (or the corresponding MP3 file doesn't exist), then I want to run a bunch of commands to convert the FLAC file to an MP3 file, and copy the tags across.
The kicker: I need to search the flac directory recursively, and create corresponding subdirectories in the mp3 directory. The directories and files can have spaces in the names, and are named in UTF-8.
And I want to use make to drive this.
I would try something along these lines
FLAC_FILES = $(shell find flac/ -type f -name '*.flac')
MP3_FILES = $(patsubst flac/%.flac, mp3/%.mp3, $(FLAC_FILES))
.PHONY: all
all: $(MP3_FILES)
mp3/%.mp3: flac/%.flac
#mkdir -p "$(#D)"
#echo convert "$<" to "$#"
A couple of quick notes for make beginners:
The # in front of the commands prevents make from printing the command before actually running it.
$(#D) is the directory part of the target file name ($#)
Make sure that the lines with shell commands in them start with a tab, not with spaces.
Even if this should handle all UTF-8 characters and stuff, it will fail at spaces in file or directory names, as make uses spaces to separate stuff in the makefiles and I am not aware of a way to work around that. So that leaves you with just a shell script, I am afraid :-/
You can define your own recursive wildcard function like this:
rwildcard=$(foreach d,$(wildcard $(1:=/*)),$(call rwildcard,$d,$2) $(filter $(subst *,%,$2),$d))
The first parameter ($1) is a list of directories, and the second ($2) is a list of patterns you want to match.
Examples:
To find all the C files in the current directory:
$(call rwildcard,.,*.c)
To find all the .c and .h files in src:
$(call rwildcard,src,*.c *.h)
This function is based on the implementation from this article, with a few improvements.
If you're using Bash 4.x, you can use a new globbing option, for example:
SHELL:=/bin/bash -O globstar
list:
#echo Flac: $(shell ls flac/**/*.flac)
#echo MP3: $(shell ls mp3/**/*.mp3)
This kind of recursive wildcard can find all the files of your interest (.flac, .mp3 or whatever). O
FWIW, I've used something like this in a Makefile:
RECURSIVE_MANIFEST = `find . -type f -print`
The example above will search from the current directory ('.') for all "plain files" ('-type f') and set the RECURSIVE_MANIFEST make variable to every file it finds. You can then use pattern substitutions to reduce this list, or alternatively, supply more arguments into find to narrow what it returns. See the man page for find.
My solution is based on the one above, uses sed instead of patsubst to mangle the output of find AND escape the spaces.
Going from flac/ to ogg/
OGGS = $(shell find flac -type f -name "*.flac" | sed 's/ /\\ /g;s/flac\//ogg\//;s/\.flac/\.ogg/' )
Caveats:
Still barfs if there are semi-colons in the filename, but they're pretty rare.
The $(#D) trick won't work (outputs gibberish), but oggenc creates directories for you!
Here's a Python script I quickly hacked together to solve the original problem: keep a compressed copy of a music library. The script will convert .m4a files (assumed to be ALAC) to AAC format, unless the AAC file already exists and is newer than the ALAC file. MP3 files in the library will be linked, since they are already compressed.
Just beware that aborting the script (ctrl-c) will leave behind a half-converted file.
I originally also wanted to write a Makefile to handle this, but since it cannot handle spaces in filenames (see the accepted answer) and because writing a bash script is guaranteed to put in me in a world of pain, Python it is. It's fairly straightforward and short, and thus should be easy to tweak to your needs.
from __future__ import print_function
import glob
import os
import subprocess
UNCOMPRESSED_DIR = 'Music'
COMPRESSED = 'compressed_'
UNCOMPRESSED_EXTS = ('m4a', ) # files to convert to lossy format
LINK_EXTS = ('mp3', ) # files to link instead of convert
for root, dirs, files in os.walk(UNCOMPRESSED_DIR):
out_root = COMPRESSED + root
if not os.path.exists(out_root):
os.mkdir(out_root)
for file in files:
file_path = os.path.join(root, file)
file_root, ext = os.path.splitext(file_path)
if ext[1:] in LINK_EXTS:
if not os.path.exists(COMPRESSED + file_path):
print('Linking {}'.format(file_path))
link_source = os.path.relpath(file_path, out_root)
os.symlink(link_source, COMPRESSED + file_path)
continue
if ext[1:] not in UNCOMPRESSED_EXTS:
print('Skipping {}'.format(file_path))
continue
out_file_path = COMPRESSED + file_path
if (os.path.exists(out_file_path)
and os.path.getctime(out_file_path) > os.path.getctime(file_path)):
print('Up to date: {}'.format(file_path))
continue
print('Converting {}'.format(file_path))
subprocess.call(['ffmpeg', '-y', '-i', file_path,
'-c:a', 'libfdk_aac', '-vbr', '4',
out_file_path])
Of course, this can be enhanced to perform the encoding in parallel. That is left as an exercise to the reader ;-)