Errors:
./themezip: line 8: unexpected EOF while looking for matching `''
./themezip: line 11: syntax error: unexpected end of file
My Code:
cat ~/scripts/script-files/repos.txt | xargs -I % sh -c git clone %' && ls
ls -d ~/themes/* > ~/scripts/script-files/dirs
lines=$(wc --lines ~/scripts/script-files/dirs)
sed s/.$// ~/scripts/script-files/dirs > ~/scripts/script-files/dirs1
paste ~/scripts/script-files/dirs1 ~/scripts/script-files/dirs > ~/scripts/script-files/dirs2
cat ~/scripts/script-files/dirs1 | xargs -I % sh -c 'zip -r -q ~/themes/% ~/themes/%/'
cat ~/scripts/script-files/dirs1 | xargs -I % sh -c 'rm -r ~/themes/%/'
rm ~/scripts/script-files/dirs*
The message should be rather clear: you are missing a '. And at the end of the file, you have not closed the last quotes, so the End Of File is a syntax error.
Finding the place where you missed a quote is sometimes tricky. shellcheck.net can help you find the place where it probably went wrong. Most Linux distributions also have a package shellcheck, which does basically the same.
Shellcheck will give some more hints, like the useless use of cat and the fact that lines is never used.
Personally, I would also put the sed pattern in single quotes, even though shellcheck does not mention it.
And, as #user2182349 stated in the comments, the quote is missing at the first line.
Related
I would like to convert and execute
if egrep -r 'my_pattern' ./template_builder
then exit 1
elif egrep -r 'my_second_pattern' ./template_builder
then exit 1
fi
in a Makefile, without success for now.
To build this:
cd /tmp;
mkdir template_builder;
echo "not_pattern" >> ./template_builder/test.txt
# Do the command at the top, nothing happens
echo "my_pattern" >> ./template_builder/test.txt
# Do the command at the top, terminal stops
touch Makefile
In a Makefile, I thought this would work :
check:
if egrep -r 'my_pattern' ./template_builder
then exit 1
elif egrep -r 'my_second_pattern' ./template_builder
then exit 1
fi
make check
if egrep -r 'my_pattern' ./template_builder
/bin/sh: -c: line 1: syntax error: unexpected end of file
make: *** [template] Error 2
How can I fix this?
Your attempt was not far from working!
Add backslashes at the end of every line, and ;s as explicit command separators (and of course use real tabs instead of the 8-space indents below):
check:
if egrep -r 'my_pattern' ./template_builder; \
then exit 1; \
elif egrep -r 'my_second_pattern' ./template_builder; \
then exit 1; \
fi
If I understand you correctly, if the directory template_builder located in /tmp does not contain a file matching the string 'my_pattern' or 'my_second_pattern', you want to exit from make with an error code.
You can achieve this with this rule in Makefile:
check:
egrep -r -v 'my_pattern' /tmp/template_builder || egrep -r -v 'my_second_pattern' /tmp/template_builder
Explanation: the first egrep is going to return an error in the case he finds a match. Due to the presence of the || operator, the second egrep will be invoked. The result of this second command will be the result that make will see. If it returns an error, the execution of make is aborted, which seems to be the behaviour you are expecting.
Caution: I edited my answer. The right boolean operator is || and not &&.
As others have already noted, make runs each separate line in a recipe in a new shell subprocess. (For the record, it uses sh out of the box, not Bash.) The trivial fix is to add a backslash to escape the newline at the end of each line which should be executed in the same shell as the next one. (You need to add a semicolon as well in some places, like before then and else and fi.) But you really want to refactor to use the facilities and idioms of make.
The default logic of make is to terminate the recipe if any line fails. So, your code can be reduced to simply
check: template_builder
! egrep -r 'my_pattern' $<
! egrep -r 'my_second_pattern' $<
The explicit exit 1 is not necessary here (negating a zero exit code produces exactly that); but if you wanted to force a particular exit code, you could do that with
egrep -r 'my_pattern' $< && exit 123 || true
Modern POSIX prefers grep -E over legacy egrep; of course, with these simple patterns, you can use just grep, or even grep -F (née fgrep).
Moreover, if you want to search for both patterns in the same set of files, it's much more efficient to search for them both at once.
check: template_builder
! egrep -e 'my_pattern' -e 'my_second_pattern' -r $<
... or combine them into a single regex my_(second_)?pattern (which requires egrep / grep -E).
Notice also how I factored out the dependency into $< and made it explicit; but you probably want to make this recipe .PHONY anyway, so that it gets executed even if nothing has changed.
(You can't directly copy/paste this code, because Stack Overflow stupidly renders the literal tabs in the markdown source as spaces.)
tag is a command line executable that allows macOS users to add a "tag" to a file.
tag -a | --add <tags> <path>... Add tags to file
I am trying to pipe a list of files in a text document but after a few days of failing badly need help on the syntax. The only clue on the description page is this:
If you plan to pipe the output of tag through xargs, you might want to
use the -0 option of each.
I have tried dozens of commands: xargs, gnu-xargs, for, while, and cannot get this to add tags to the files in the list. Researching the net I thought maybe there was an issue with line endings.
I installed dos2unix and ran it on the file to fix possible line ending issues, but that didn't appear to resolve anything.
If you cd into the directory of the files you are attempting to tag you don't have to use the complete path to the file.
$ gxargs -0 -t --arg-file="/Users/john/Desktop/diffremove.txt" | tag -0 -a red
echo '13.Hours.The.Secret.Soldiers.of.Benghazi.2016.1080p.BluRay.x265'$'\n''1941.1979.DC.1080p.BluRay.x265'$'\n'...
Not understanding how xargs passes the lines I thought I need to put a variable in position after the command where it's looking for the file tag -0 -a red <variable here>
I tried:
$ for i in $(/Users/john/Desktop/diffremove.txt) do `tag -a red "$1"`
I installed gnu xargs and tried this command.
$ </Users/john/Desktop/diffremove.txt gxargs -0 sh -c 'for arg do tag -0 -a red "$arg"'
EDIT:
Trying this command results in no errors but the files aren't tagged.
$ </Users/john/Desktop/diffremove.txt xargs -0 sh -c 'for arg do `tag -0 -a red '$arg'`;done'
Try this
xargs -d '\n' -a /Users/john/Desktop/diffremove.txt -I % tag -a red %
here we use xargs to read from file (using -a), using replacing string with (using %) and execute command tag -a red {filename}. You may need to add -d '\n' (setting delimeter to newline) to split strings.
However, classic way to read and process lines in file is using shell builtin command read:
$ while IFS= read -r p; do tag -a red $p; done < /Users/john/Desktop/diffremove.txt
,where IFS= part can be skipped if you don't have spaces in file.
This question already has answers here:
Run script on multiple files
(3 answers)
Closed 3 years ago.
I'm very new to bash. I have ten text files that I want to edit with the same line of code.
#!/bin/bash
sed -i -e 's/.\{6\}/&\n/g' -e 's/edit/edit2/g' | tr -d "\n" | sed 's/edit2/edit/g'| grep -o "here.*there" | sed -r '/^.{,100}$/d'
< files 1-10
I know I could use sed -f sed.sh <file1 >file1 but that only works with sed commands and it only works one file at a time?
Do I have to run a loop?
There's some great existing answers on the Unix stack exchange that help deal with your problem. Specifically, from this post, they use a loop to recursively loop through all the files in a particular directory, as follows:
( shopt -s globstar dotglob;
for file in **; do
if [[ -f $file ]] && [[ -w $file ]]; then
sed -i -- 's/foo/bar/g' "$file"
fi
done
)
Note the line, shopt -s globstar dotglob;, which allows us to use globbing patterns in the for loop. We also enclose the code in brackets, to prevent the shopt -s globstar dotglob; line option from becoming a global setting.
If you would like to apply this example to your file, you can just place your files in the current directory, and the code would probably look something like this:
( shopt -s globstar dotglob;
for file in **; do
if [[ -f $file ]] && [[ -w $file ]]; then
sed -i -e 's/.\{6\}/&\n/g' -e 's/edit/edit2/g' | tr -d "\n" | sed 's/edit2/edit/g' | grep -o "here.*there" | sed -r '/^.{,100}$/d' "$file"
fi
done
)
Note that we have placed a "$file" variable beside each of the seds that you used in your code, this replaces the name of the file for each command.
There is another example given in the code that allows you to pick which files to run on, rather than all the files in a directory, which you can also re-purpose for your code, as given here:
( shopt -s globstar dotglob
sed -i -- 's/foo/bar/g' **baz*
sed -i -- 's/foo/bar/g' **.baz
)
To answer your question of doing a loop on each line, you will need to put a loop for each line inside your for loop, like so:
while read line ; do
: sed -i -e 's/.\{6\}/&\n/g' -e 's/edit/edit2/g' | tr -d "\n" | sed 's/edit2/edit/g' | grep -o "here.*there" | sed -r '/^.{,100}$/d' "$line”
done
)
Although the for loop can be useful for dealing with files in recursive directories, I would recommend against also using another loop to grab lines, since it muddies your code, and it’s possible there is a better way to do it without parsing line by line.
The linked question is a fairly complete guide to many of the cases you may come across, and is also worth a read if you want to learn more.
Hope that helps!
You could use a for loop.
You could use the tool parallel.
Example
Create a set of test files using a for-loop
mkdir -p /tmp/so58333536
cd /tmp/so58333536
for i in 1.txt 2.txt 3.txt 4.txt 5.txt;do echo "The answer is 41" > $i;done
cat /tmp/so58333536/*
Now correct your mistake using parallel [1].
mkdir /tmp/so58333536.new
ls /tmp/so58333536/* |parallel "sed 's/41/42/' {} > /tmp/so58333536.new/{/}"
cat /tmp/so58333536.new/*
{}:: refers to the current file
{/}:: refers to name of the current file (path is removed)
Reads: List all files in so58333536 and apply the following sed command to each file and write the output to so58333536.new.
[1] Another option is to use sed -i for in-place editing.
Be very carefull with this!! Mistakes can cause serious damages!
# !! Do not use -i option regularly !!
ls /tmp/so58333536/* |parallel "sed -i 's/41/42/'"
I suppose the answer here might be trivial, but I it might require some intricate bash knowledge. I have been browsing bash docs for a few hours now and can't seem to find the answer.
I'm working on a python repository, and came up with a simple script to lint only the files that differ between the current branch and master. Here's the minimal working example, extracted from said script (lint.sh):
#!/bin/bash
paths=$(git diff --name-only -r origin/master...HEAD | grep \.py$)
flake8 $paths
For testing purposes, let's say I only committed one file, bad.py, with the following contents:
hello
there
The expected output of bash lint.sh is:
bad.py:1:1: F821 undefined name 'hello'
bad.py:2:1: F821 undefined name 'there'
However, the output is empty. When run in debug mode, bash shows the following commands:
++ git diff --name-only -r origin/master...HEAD
++ grep '.py$'
+ paths='bad.py'
+ flake8 'bad.py'
Which is what I expect. Also, when I simply run flake8 bad.py, the output is as expected.
I expect this might have something to do with parameter passing which varies between different bash versions. The output of bash --version:
GNU bash, version 4.4.23(1)-release (x86_64-apple-darwin17.5.0)
I will appreciate all insights
Very sorry this isn't exactly an answer, but it surely didn't fit in a comment!
The hint here to me is the following:
+ paths='bad.py'
+ flake8 'bad.py'
In my execution of the same script, I get the following:
$ bash -x lint.sh
++ git diff --name-only -r origin/master...HEAD
++ grep '.py$'
+ paths=bar.py
+ flake8 bar.py
bar.py:1:1: F821 undefined name 'hello'
bar.py:2:1: F821 undefined name 'world'
Notice here how my output does not contain quotes around the filename or the assignment. bash won't usually add quotes unless they are necessary. What this tells me is there's probably some sort of control character in that string (my best guess is either colors or \b + some other characters (this might be one of the few cases where a screenshot is actually helpful!)).
Here's one way that I was able to reproduce your findings:
mkdir -p bin
cat > bin/grep << EOF
#!/usr/bin/env bash
exec /bin/grep --color=always "\$#"
EOF
chmod +x bin/grep
# simulate having this `grep` on your path
PATH=$PWD/bin:$PATH bash -x lint.sh
(and while this seems like an odd thing to do, in the past I've put my own grep in ~/bin so I could add --line-buffered --color=auto now that GREP_OPTIONS is deprecated -- one might erroneously add --color=always and have it work... for the most part). Today I use an alias instead since I ran into sharp edges even with that.
The output in that case matches yours above:
$ PATH=$PWD/bin:$PATH bash -x lint.sh
++ git diff --name-only -r origin/master...HEAD
++ grep '.py$'
+ paths='bar.py'
+ flake8 'bar.py'
But the tricky hint is in the highlighting
addendum
While unrelated to your problem, here's probably a better way to accomplish what you want:
# if you have GNU xargs
git diff -z --name-only origin/master...HEAD -- '*.py' | xargs --null --no-run-if-empty flake8
# if you need to be more portable (I see you're probably on macos)
git diff -z --name-only origin/master...HEAD -- '*.py' | xargs -0 flake8 /dev/null
Explanation of the different parts:
git diff -z: output filenames with null bytes delimiting. This prevents splicing if filenames contain spaces or other special characters
xargs --null: split the input by null bytes when splatting arguments
xargs --no-run-if-empty: don't run the executable at all if there's no arguments (this is a GNU extension)
xargs -0: same as xargs --null, however if you're stuck with non-GNU xargs you won't have access to the long options
flake8 /dev/null: this is a sneaky trick, since there's no "no run if empty" option to bsd xargs, it's always going to invoke flake8. If flake8 gets invoked with zero arguments, it defaults to recursing your current working directory (and linting all your files). By putting /dev/null at the beginning this prevents this behaviour and instead lints an empty file!
Addendum 2, you probably might want to consider using a git hooks framework to handle all of this for you, I maintain pre-commit which aims to smooth out a lot of the rough edges around git (such as this one!).
I'd like to be able to comment out a single flag in a one-line command. Bash only seems to have from # till end-of-line comments. I'm looking at tricks like:
ls -l $([ ] && -F is turned off) -a /etc
It's ugly, but better than nothing. Is there a better way?
The following seems to work, but I'm not sure whether it is portable:
ls -l `# -F is turned off` -a /etc
My preferred is:
Commenting in a Bash script
This will have some overhead, but technically it does answer your question
echo abc `#put your comment here` \
def `#another chance for a comment` \
xyz etc
And for pipelines specifically, there is a cleaner solution with no overhead
echo abc | # normal comment OK here
tr a-z A-Z | # another normal comment OK here
sort | # the pipelines are automatically continued
uniq # final comment
How to put a line comment for a multi-line command
I find it easiest (and most readable) to just copy the line and comment out the original version:
#Old version of ls:
#ls -l $([ ] && -F is turned off) -a /etc
ls -l -a /etc
$(: ...) is a little less ugly, but still not good.
Here's my solution for inline comments in between multiple piped commands.
Example uncommented code:
#!/bin/sh
cat input.txt \
| grep something \
| sort -r
Solution for a pipe comment (using a helper function):
#!/bin/sh
pipe_comment() {
cat -
}
cat input.txt \
| pipe_comment "filter down to lines that contain the word: something" \
| grep something \
| pipe_comment "reverse sort what is left" \
| sort -r
Or if you prefer, here's the same solution without the helper function, but it's a little messier:
#!/bin/sh
cat input.txt \
| cat - `: filter down to lines that contain the word: something` \
| grep something \
| cat - `: reverse sort what is left` \
| sort -r
Most commands allow args to come in any order. Just move the commented flags to the end of the line:
ls -l -a /etc # -F is turned off
Then to turn it back on, just uncomment and remove the text:
ls -l -a /etc -F
How about storing it in a variable?
#extraargs=-F
ls -l $extraargs -a /etc
If you know a variable is empty, you could use it as a comment. Of course if it is not empty it will mess up your command.
ls -l ${1# -F is turned off} -a /etc
§ 10.2. Parameter Substitution
For disabling a part of a command like a && b, I simply created an empty script x which is on path, so I can do things like:
mvn install && runProject
when I need to build, and
x mvn install && runProject
when not (using Ctrl + A and Ctrl + E to move to the beginning and end).
As noted in comments, another way to do that is Bash built-in : instead of x:
$ : Hello world, how are you? && echo "Fine."
Fine.
It seems that $(...) doesn't survive from ps -ef.
My scenario is that I want to have a dummy param that can be used to identify the very process. Mostly I use this method, but the method is not workable everywhere. For example, python program.py would be like
mkdir -p MyProgramTag;python MyProgramTag/../program.py
The MyProgramTag would be the tag for identifying the process started.