I am currently in the process of writing quite a large document, that I divided into multiple chapters. What I chose to do is to put each chapter in a different file, and then compile them with a makefile into one big document. When I type make get in the terminal, the .tex docs appear with the right name in my folder but completely blank.
Here is my makefile :
pdf:
pdflatex --shell-escape file.tex
clean:
rm -f chapter1.tex
rm -f chapter2.tex
get:
sed -e '/BEGIN_BOOK/,/END_BOOK/!d' ../01-chapter1/chapter1.tex> chapter1.tex
sed -e '/BEGIN_BOOK/,/END_BOOK/!d' ../02-chapter2/chapter2.tex> chapter2.tex
diff:
git latexdiff HEAD -- --main file.tex --latexopt "--shell-escape"
But if I replace these blank ones with the correct files, the make command works properly and I get what I want.
Thank you in advance
Related
I have video files with this structure : GX**#### where #### is the number of the video.
But sometimes videos are splitted in 2 or 3 files, for instance : GX01#### and GX02####
My problem is that to organise my folders I prefer rename them like this : GX####-1 and GX####-2.
So i ask you the question if I can create a script/command to rename automatically my files to do this ? I can use .bat windows files or .sh linux files. My main problem is that i don't know a command to rename files simply (i saw many solutions which rewrite the entire file). May be if you know just this I will can do my script.
Thanks you in advance.
You can loop through the files and use PE parameter expansion to slice and jumble the strings and mv to rename the files.
Here is an example, first let us create an empty files using touch, but first we need to create a new directory (folder as you call it) and go into that newly created directory, using mkdir and cd.
mkdir newdirectory && cd newdirectory
Now create the empty files using touch.
touch GX01#### GX02#### GX03#### GX04####
Now check the empty files using ls
ls *
The output should be like
GX01#### GX02#### GX03#### GX04####
Now that we have created empty files, we can now do the renaming part.
for file in GX*; do
file1="${file#*??}"
file2=${file1#*??}
file3=${file1%*$file2}
echo mv -v "$file" "${file%*$file1}${file2}"-"$file3"
done
The output should be on stdout (which is your screen) because of the echo.
mv -v GX01#### GX####-01
mv -v GX02#### GX####-02
mv -v GX03#### GX####-03
mv -v GX04#### GX####-04
If you're satisfied with what you see then remove the echo, so mv can actually rename the files. To show how the slicing is done here is an example.
file=GX01####
file1="${file#*??}"
file2=${file1#*??}
file3="${file1%*$file2}"
printf '%s\n' "$file1" "$file2" "$file3"
The output should be something like.
01####
####
01
The ? means a single string from the shell.
A word of caution and advice, make a backup of the files you want to edit just in case the shell made a mistake while you're renaming the files. :-)
Also you should probably use shell globing like nullglob see
http://mywiki.wooledge.org/glob#nullglob
See man bash and look for Parameter Expansion.
PAGER='less +/^[[:space:]]*parameter\ expansion' man bash
Some online resources, with detailed explanation and examples.
http://mywiki.wooledge.org/BashFAQ/073
https://wiki.bash-hackers.org/syntax/pe
You could echo the original and new file names:
for f in GX*; do
echo "$f" $(sed 's/^GX\(..\)\(.*\)/GX\2-\1/' <<< "$f")
done
which should output:
GX01#### GX####-01
GX02#### GX####-02
then use mv -n instead of echo to rename the files.
Suppose I have a Makefile like this
B1.txt: A1.txt
python big_long_program.py A1.txt > $#
correct1.txt: B1.txt reference.txt
diff -q B1.txt reference.txt
touch $#
Then the output when I make correct1.txt is pretty well what I would expect:
python big_long_program.py A1.txt > B1.txt
diff -q B1.txt reference.txt
touch correct1.txt
Now if I have lots of files, B1.txt, B2.txt, B3.txt etc, so create an implicit rule:
B%.txt: A%.txt
python big_long_program.py A$*.txt > $#
correct%.txt: B%.txt reference.txt
diff -q B$*.txt reference.txt
touch $#
Instead this happens when I make correct1.txt:
python big_long_program.py A1.txt > B1.txt
diff -q B1.txt reference.txt
touch correct1.txt
rm B1.txt
i.e. there difference is that now the file B1.txt has been deleted, which in many cases is really bad.
So why are implicit rules different? Or am I doing something wrong?
You are not doing anything wrong. The behavior you observe and analyze is documented in 10.4 Chains of Implicit Rules. It states that intermediate files are indeed treated differently.
The second difference is that if make does create b in order to update
something else, it deletes b later on after it is no longer needed.
Therefore, an intermediate file which did not exist before make also
does not exist after make. make reports the deletion to you by
printing a rm -f command showing which file it is deleting.
The documentation does not explicitly explain why it behaves like this. Looking in the file ChangeLog.1, there is a reference to the remove_intermediates function as far back as 1988. At that time, disk space was expensive and at a premium.
If you do not want this behavior, mention the targets you want to keep somewhere in the makefile as an explicit prerequisite or target or use the .PRECIOUS or the .SECONDARY special built-in targets for that.
With thanks to MadScientist for the additional comments, see below.
I want to read a file called metafile inside a makefile.
metafile looks something like this:
file1
file2
file3
I need to read this metafile inside my makefile line by line and check if the files mentioned inside exists or not and only print names of files which exists.
I've tried a few things without success. like:
FILE=`cat metafile`
for line in $(FILE); if [ -e $${line} ]; then echo $${line} fi; done;
You can put an arbitrary piece of shell script in a target. Keeping the file's contents in a Makefile variable does not make any sense to me, unless you also need the data in other targets for other reasons. (If so, you cannot use backticks, anyway.)
target:
#while read -r file; do \
test -e "$$file" && echo "$$file"; \
done <metafile
For what it's worth, the while loop is a safer and more idiomatic way to loop over a file's lines in a shell script than the for loop with backticks, even though you see that a lot.
The # prevents Make from echoing the shell script commands; take that out if for some reason you need to see them. In fact, I recommend against using this, especially while you are debugging -- use make -s to have make run silently once you are confident your recipe works correctly.
A more idiomatic way to do this in a Makefile is to have a target depend on these files, and use Make's own logic:
target: file1 file2 file3
#echo $(filter-out $?,$^)
This is GNU Make syntax; it might get more complex if you want to be portable to other Make flavors (to the point where maybe the shell script is preferable after all). It will echo everything on one line, but if you need separate lines, that should be a trivial fix.
I would simply build a small auxiliary Makefile snippet and include the dependencies:
target: target.d
target.d: metafile
sed 's/^/target: /' $< >$#
include target.d
This builds a small list of dependencies so you don't need to list them in the target: dependencies explicitly; so instead of file1 file2 file3 in the recipe above, the dependencies would live in the generated target.d which would contain
target: file1
target: file2
target: file3
You need to filter out the dependency on target.d (or leave it undeclared; I believe GNU Make should cope).
It's been a while since I've used make, so bear with me...
I've got a directory, flac, containing .FLAC files. I've got a corresponding directory, mp3 containing MP3 files. If a FLAC file is newer than the corresponding MP3 file (or the corresponding MP3 file doesn't exist), then I want to run a bunch of commands to convert the FLAC file to an MP3 file, and copy the tags across.
The kicker: I need to search the flac directory recursively, and create corresponding subdirectories in the mp3 directory. The directories and files can have spaces in the names, and are named in UTF-8.
And I want to use make to drive this.
I would try something along these lines
FLAC_FILES = $(shell find flac/ -type f -name '*.flac')
MP3_FILES = $(patsubst flac/%.flac, mp3/%.mp3, $(FLAC_FILES))
.PHONY: all
all: $(MP3_FILES)
mp3/%.mp3: flac/%.flac
#mkdir -p "$(#D)"
#echo convert "$<" to "$#"
A couple of quick notes for make beginners:
The # in front of the commands prevents make from printing the command before actually running it.
$(#D) is the directory part of the target file name ($#)
Make sure that the lines with shell commands in them start with a tab, not with spaces.
Even if this should handle all UTF-8 characters and stuff, it will fail at spaces in file or directory names, as make uses spaces to separate stuff in the makefiles and I am not aware of a way to work around that. So that leaves you with just a shell script, I am afraid :-/
You can define your own recursive wildcard function like this:
rwildcard=$(foreach d,$(wildcard $(1:=/*)),$(call rwildcard,$d,$2) $(filter $(subst *,%,$2),$d))
The first parameter ($1) is a list of directories, and the second ($2) is a list of patterns you want to match.
Examples:
To find all the C files in the current directory:
$(call rwildcard,.,*.c)
To find all the .c and .h files in src:
$(call rwildcard,src,*.c *.h)
This function is based on the implementation from this article, with a few improvements.
If you're using Bash 4.x, you can use a new globbing option, for example:
SHELL:=/bin/bash -O globstar
list:
#echo Flac: $(shell ls flac/**/*.flac)
#echo MP3: $(shell ls mp3/**/*.mp3)
This kind of recursive wildcard can find all the files of your interest (.flac, .mp3 or whatever). O
FWIW, I've used something like this in a Makefile:
RECURSIVE_MANIFEST = `find . -type f -print`
The example above will search from the current directory ('.') for all "plain files" ('-type f') and set the RECURSIVE_MANIFEST make variable to every file it finds. You can then use pattern substitutions to reduce this list, or alternatively, supply more arguments into find to narrow what it returns. See the man page for find.
My solution is based on the one above, uses sed instead of patsubst to mangle the output of find AND escape the spaces.
Going from flac/ to ogg/
OGGS = $(shell find flac -type f -name "*.flac" | sed 's/ /\\ /g;s/flac\//ogg\//;s/\.flac/\.ogg/' )
Caveats:
Still barfs if there are semi-colons in the filename, but they're pretty rare.
The $(#D) trick won't work (outputs gibberish), but oggenc creates directories for you!
Here's a Python script I quickly hacked together to solve the original problem: keep a compressed copy of a music library. The script will convert .m4a files (assumed to be ALAC) to AAC format, unless the AAC file already exists and is newer than the ALAC file. MP3 files in the library will be linked, since they are already compressed.
Just beware that aborting the script (ctrl-c) will leave behind a half-converted file.
I originally also wanted to write a Makefile to handle this, but since it cannot handle spaces in filenames (see the accepted answer) and because writing a bash script is guaranteed to put in me in a world of pain, Python it is. It's fairly straightforward and short, and thus should be easy to tweak to your needs.
from __future__ import print_function
import glob
import os
import subprocess
UNCOMPRESSED_DIR = 'Music'
COMPRESSED = 'compressed_'
UNCOMPRESSED_EXTS = ('m4a', ) # files to convert to lossy format
LINK_EXTS = ('mp3', ) # files to link instead of convert
for root, dirs, files in os.walk(UNCOMPRESSED_DIR):
out_root = COMPRESSED + root
if not os.path.exists(out_root):
os.mkdir(out_root)
for file in files:
file_path = os.path.join(root, file)
file_root, ext = os.path.splitext(file_path)
if ext[1:] in LINK_EXTS:
if not os.path.exists(COMPRESSED + file_path):
print('Linking {}'.format(file_path))
link_source = os.path.relpath(file_path, out_root)
os.symlink(link_source, COMPRESSED + file_path)
continue
if ext[1:] not in UNCOMPRESSED_EXTS:
print('Skipping {}'.format(file_path))
continue
out_file_path = COMPRESSED + file_path
if (os.path.exists(out_file_path)
and os.path.getctime(out_file_path) > os.path.getctime(file_path)):
print('Up to date: {}'.format(file_path))
continue
print('Converting {}'.format(file_path))
subprocess.call(['ffmpeg', '-y', '-i', file_path,
'-c:a', 'libfdk_aac', '-vbr', '4',
out_file_path])
Of course, this can be enhanced to perform the encoding in parallel. That is left as an exercise to the reader ;-)
I'm using a makefile to automate some document generation. I have several documents in a directory, and one of my makefile rules will generate an index page of those files. The list of files itself is loaded on the fly using list := $(shell ls documents/*.txt) so I don't have to bother manually editing the makefile every time I add, delete, or rename a document. Naturally, I want the index-generation rule to trigger when number/title of files in the documents directory changes, but I don't know how to set up the prerequisites to work in this way.
I could use .PHONY or something similar to force the index-generation to run all the time, but I'd rather not waste the cycles. I tried piping ls to a file list.txt and using that as a prerequisite for my index-generation rule, but that would require either editing list.txt manually (trying to avoid it), or auto-generating it in the makefile (this changes the creation time, so I can't use list.txt in the prerequisite because it would trigger the rule every time).
If you need a dependency on the number of files, then... why not just depend on the number itself? The number will be represented as a dummy file that is created when the specified nubmer of files is in the documents directory.
NUMBER=$(shell ls documents/*.txt | wc -l).files
# This yields name like 2.files, 3.files, etc...
# .PHONY $(NUMBER) -- NOT a phony target!
$(NUMBER):
rm *.files # Remove previous trigger
touch $(NUMBER)
index.txt: $(NUMBER)
...generate index.txt...
While number of files is one property to track, instead you may depend on a hash of a directory listing. It's very unlikely that hash function will be the same for two listings that occur in your workflow. Here's an example:
NUMBER=$(shell ls -l documents/*.txt | md5sum | sed 's/[[:space:]].*//').files
Note using -l -- this way you'll depend on full listing of files, which includes modification time, sizes and file names. Bu if you don't need it, you may drop the option.
Note: sed was needed because on my system md5sum yields some stuff after the hash-sum itself.
You can put a dependency on the directory itself for the list file, e.g.
list.txt: documents
ls documents/*.txt 2>/dev/null > $# || true
Every time you add or remove a file in the documents directory, the directory's timestamp will be altered and make will do the right thing.
Here's a solution that updates the index if and only if the set of files has changed:
list.txt.tmp: documents
ls $</*.txt > $#
list.txt: list.txt.tmp
cmp -s $< $# || cp $< $#
index.txt: list.txt
...generate index.txt...
Thanks to the "cmp || cp", the ctime of "list.txt" does not change unless the output of the "ls" has changed.