one case:
I wrote script on tsch that invoke other script on python.
When I invoke python script from cmd , it is OK.
When I invoke test script on tsch then I get error: Argument list too long
Another case:
git grep -e "alex" -- `git ls-files | grep -v 'bin'`
I also get error: Argument list too long.
What can problem and How to solve it ?
Updated Answer
I'm not familiar with the specific git commands you are using and you don't seem to be replying sensibly to the questions in the comments either. I guess you probably want something like this though:
git ls-files | grep -v 'bin' | xargs -L 128 git grep -e "alex" --
Original Answer
The classic way to solve "error: Argument list too long" is with xargs. It can be used to repeatedly call a script whose name you provide, or echo if you don't provide one, with a limited number of arguments till the arguments are all consumed.
So, imagine you have a million files in a directory, then ls * will fail, however a simple ls will work. So, we can put that to use with:
ls | xargs -L 128
which will call echo (because we didn't provide a utility name) repeatedly with 128 filenames at a time till all are echoed.
So, you could do:
ls | xargs -L 128 yourScript.py
to call your Python script repeatedly with 128 filenames at a time. Of course you may be doing something completely different and incompatible with this technique but your answers are not very helpful so far...
for somebody comes here who need to do something like this:
./shell_script.sh param1
but it raises error Argument list too long from shell script of param1.
I just run into this, and fix it by a workaround of using the shell variable.
# calling the PARAM1 instead of $1 in code of shell_script.sh
export PARAM1=param1 ./shell_script.sh
an example of a ruby version of transferring string to nodejs:
ENV["PARAM1"]="a_bunch_of_test_string_as_longer_as_you_can"
`node node_script.sh`
var param1 = process.env.PARAM1;
console.log(param1);
Related
Trying to compare the process list before and after running a bash script of tests. Having trouble, since ps returns 1, and I'm not sure how to compare the before and after when I have them.
Ideally, it would look something like this. Forgive the crude pseudo-code:
run-tests:
ps -x
export before=$?
# run tests and scripts
ps -x
export after=$?
# compare before and after
Suggests and advice appreciated.
I'm assuming you want to count the number of running processes before and after (your question wasn't overly clear on that). If so, you can pipe ps into wc:
export before=`ps --no-headers | wc -l`
-- EDIT ---
I reread the question, and it may be that you're looking for the actual processes that differ. If that's the case, then, you can capture the output in variables and compare those:
target:
# before=$$(ps --no-headers); \
run test; \
after=$$(ps --no-headers); \
echo "differing processes:"; \
comm -3 <(echo "$before") <(echo "$after")
A few quick notes on this: I concatenated all the lines using \'s as you mentioned you used makefiles, and the scope of a variable is only the recipe line in which it's defined. By concatenating the lines, the variables have a scope of the whole recipe.
I used double $$ as your original post suggested a makefile, and a makefile $$ will expand to a single $ in the bash code to be run.
Doing var=$(command) in bash assigns var the output of command
I used the <() convention which is specific to bash. This lets you treat the output of a command as file, without having to actually create a file. Notice that I put quotes around the variables -- this is required, otherwise the bash will ignore newlines when expanding the variable.
I have a directory "Main Dir" and I want to write a script, which will get 2 parameters: sorted_by_date , that will find in the directory the id-worker directory (it does exist) and in it, from a file called "sent.txt", it will print results-num (an integer) strokes sorted by date.
I'm a begginer in bash (have knowledge and skills mainly in C), and I stil didn't saw how to write scripts, but I've tried to do something from a little commands I learned and from a little search in the internet.
Can somebody help a newbie like me in my first script-writing ?
I'll paste here my first try:
#!/bin/bash
id_worker = "$1"
results_num = "$2"
sort -k3 -t "./Main Dir/id_worker/sent.text"
head -n+3 $results_num
I'm going to go out on a limb here, and assume your sort command is producing the information you want from the id_worker sent.txt file and that you are talking about the number of lines you want when you say strokes. Given the extended discussion in the comments, that is about the only thing I see that makes sense.
With that in mind, you were not that far off in your first attempt. What you needed to do to fix the sort command was to dereference your id_worker with $ to get the value you passed. In bash you assign variables as id_worker="something", but to get the value back, you must precede the variable with a $, just as you see with your id_worker="$1". NOTE: there are NO spaces allowed on either side of the '=' sign in bash. Putting that together, it looks like you intended:
sort -k3 -t "./Main Dir/$id_worker/sent.text"
Where you are beginning in the directory above Main Dir running your script because you have given a relative path "./Main Dir/stuff".
Now if you want to limit the number of lines to the first results_num lines of the sorted output, then you can use head, but you need to remove the "+" sign (which is only relevant with the tail command). To use it with the sorted output, you mustpipe the results of sort to head using the '|' pipe character. For example:
sort -k3 -t "./Main Dir/$id_worker/sent.text" | head -n $results_num
Putting all of the pieces that I think you intended, and including a short check to make sure both id_worker and results_num are given on the command line, you would end up with something like:
#!/bin/bash
## verify both arguments given
[ -z $1 -o -z $2 ] && {
printf "error: insufficient input. usage: %s worker num\n" "${0##*/}"
exit 1
}
id_worker="$1"
results_num="$2"
## pipe the results of sort to head to print first $results_num lines
sort -k3 -t "./Main Dir/$id_worker/sent.text" | head -n $results_num
Note: if you are having trouble with your script, run it with:
bash -x scriptname id_worker results_num
to enable line-by-line debugging output from bash. Let me know if I have not understood what you were saying or if the results are not what you intended. There are several ways of approaching this problem, but I do need to clearly understand what you want to go further. Good luck.
I am trying to cleanly, without errors/quirks, open multiple files via command line in the vim text editor. I am using bash as my shell. Specifically, I want to open the first 23 files in the current working directory. My initial command was:
$ ls | head -23 | xargs vim
But when I do this, I get the following error message before all the files open:
Vim: Warning: Input is not from a terminal
and no new text is shown in the terminal after vim exits. I have to blindly do a reset in order to get a normal terminal back, apart from opening a new one.
This seems to be discussed here: using xargs vim with gnu screen, and: Why does "locate filename | xargs vim" cause strange terminal behaviour?
Since the warning occurs, xargs seems to be a no-no with vim. (Unless you do some convoluted thing using subshells and input/output redirection which I'm not too interested in as a frequent command. And using an alias/function... meh.)
The solution seemed to be to use bash's command substitution. So I tried:
$ vim $(ls | head -23)
But the files have spaces and parentheses in them, in this format:
surname, firstname(email).txt
So what the shell then does, which also is the result in the xarg case, is provide surname, and firstname(email).txt as two separate command arguments, leaving me in vim with at least twice the number of files I wanted to open, and none of the files I actually wanted to open.
So, I figure I need to escape the file names somehow. I tried to quote the command:
$ vim "$(ls | head -23)"
Then the shell concatenates the entire output from the substitution and provides that as a single command argument, so I'm left with a super-long file name which is also not what I want.
I've also tried to work with the -exec, -print, -print0 and -printf options of find, various things with arrays in bash, and probably some things I can't remember. I'm at a loss right now.
What can I do to use file names that come from a command, as separate command arguments and shell-quoted so they actually work?
Thanks for any and all help!
Here's an array-based solution:
fileset=(*)
vim "${fileset[#]:0:23}"
xargs -a <(ls | head -23) -d '\n' vim
-a tells xargs to read arguments from the named file instead of stdin. <(...) lets us pass the output of the ls/head pipeline where a filename is expected.
I am, like many non-engineers or non-mathematicians who try writing algorithms, an intuitive. My exact psychological typology makes it quite difficult for me to learn anything serious like computers or math. Generally, I prefer audio, because I can engage my imagination more effectively in the learning process.
That said, I am trying to write a shell script that will help me master Linux. To that end, I copied and pasted a list of Linux commands from the O'Reilly website's index to the book Python In a Nutshell. I doubt they'll mind, and I thank them for providing it. These are the textfile `massivelistoflinuxcommands,' not included fully below in order to save space...
OK, now comes the fun part. How do I get this script to work?
#/bin/sh
read -d 'massivelistoflinuxcommands' commands <<EOF
accept
bison
bzcmp
bzdiff
bzgrep
bzip2
bzless
bzmore
c++
lastb
lastlog
strace
strfile
zmore
znew
EOF
for i in $commands
do
$i --help | less | cat > masterlinuxnow
text2wave masterlinuxnow -o ml.wav
done
It really helps when you include error messages or specific ways that something deviates from expected behavior.
However, your problem is here:
read -d 'massivelistoflinuxcommands' commands <<EOF
It should be:
read -d '' commands <<EOF
The delimiter to read causes it to stop at the first character it finds that matches the first character in the string, so it stops at "bzc" because the next character is "m" which matches the "m" at the beginning of "massive..."
Also, I have no idea what this is supposed to do:
$i --help | less | cat > masterlinuxnow
but it probably should be:
$i --help > masterlinuxnow
However, you should be able to pipe directly into text2wave and skip creating an intermediate file:
$i --help | text2wave -o ml.wav
Also, you may want to prevent each file from overwriting the previous one:
$i --help | text2wave -o ml-$i.wav
That will create files named like "ml-accept.wav" and "ml-bison.wav".
I would point out that if you're learning Linux commands, you should prioritize them by frequency of use and/or applicability to a beginner. For example, you probably won't be using bison right away`.
The first problem here is that not every command has a --help option!! In fact the very first command, accept, has no such option! A better approach might be executing man on each command since a manual page is more likely to exist for each of the commands. Thus change;
$i --help | less | cat > masterlinuxnow
to
man $i >> masterlinuxnow
note that it is essential you use the append output operator ">>" instead of the create output operator ">" in this loop. Using the create output operator will recreate the file "masterlinuxnow" on each iteration thus containing only the output of the last "man $i" processed.
you also need to worry about whether the command exists on your version of linux (many commands are not included in the standard distribution or may have different names). Thus you probably want something more like this where the -n in the head command should be replace by the number of lines you want, so if you want only the first 2 lines of the --help output you would replace -n with -2:
if [ $(which $i) ]
then
$i --help | head -n >> masterlinuxnow
fi
and instead of the read command, simply define the variable commands like so:
commands="
bison
bzcmp
bzdiff
bzgrep
bzip2
bzless
bzmore
c++
lastb
lastlog
strace
strfile
zmore
znew
"
Putting this all together, the following script works quite nicely:
commands="
bison
bzcmp
bzdiff
bzgrep
bzip2
bzless
bzmore
c++
lastb
lastlog
strace
strfile
zmore
znew
"
for i in $commands
do
if [ $(which $i) ]
then
$i --help | head -1 >> masterlinuxnow 2>/dev/null
fi
done
You're going to learn to use Linux by listening to help descriptions? I really think that's a bad idea.
Those help commands usually list every obscure option to a command, including many that you will never use-- especially as a beginner.
A guided tutorial or book would be much better. It would only present the commands and options that will be most useful. For example, that list of commands you gave has many that I don't know-- and I've been using Linux/Unix extensively for 10 years.
I'm new to bash scripts (and the *nix shell altogether) but I'm trying to write this script to make grepping a codebase easier.
I have written this
#!/bin/bash
args=("$#");
for arg in args
grep arg * */* */*/* */*/*/* */*/*/*/*;
done
when I try to run it, this is what happens:
~/Work/richmond $ ./f.sh "\$_REQUEST\['a'\]"
./f.sh: line 4: syntax error near unexpected token `grep'
./f.sh: line 4: ` grep arg * */* */*/* */*/*/* */*/*/*/*;'
~/Work/richmond $
How do I do this properly?
And, I think a more important question is, how can I make grep recurse through subdirectories properly like this?
Any other tips and/or pitfalls with shell scripting and using bash in general would also be appreciated.
The syntax error is because you're missing do. As for searching recursively if your grep has the -R option you would do:
#!/bin/bash
for arg in "$#"; do
grep -R "$arg" *
done
Otherwise you could use find:
#!/bin/bash
for arg in "$#"; do
find . -exec grep "$arg" {} +
done
In the latter example, find will execute grep and replace the {} braces with the file names it finds, starting in the current directory ..
(Notice that I also changed arg to "$arg". You need the dollar sign to get the variable's value, and the quotes tell the shell to treat its value as one big word, even if $arg contains spaces or newlines.)
On recusive grepping:
Depending on your grep version, you can pass -R to your grep command to have it search Recursively (in subdirectories).
The best solution is stated above, but try putting your statement in back ticks:
`grep ...`
You should use 'find' plus 'xargs' to do the file searching.
for arg in "$#"
do
find . -type f -print0 | xargs -0 grep "$arg" /dev/null
done
The '-print0' and '-0' options assume you're using GNU grep and ensure that the script works even if there are spaces or other unexpected characters in your path names. Using xargs like this is more efficient than having find execute it for each file; the /dev/null appears in the argument list so grep always reports the name of the file containing the match.
You might decide to simplify life - perhaps - by combining all the searches into one using either egrep or grep -E. An optimization would be to capture the output from find once and then feed that to xargs on each iteration.
Have a look at the findrepo script which may give you some pointers
If you just want a better grep and don't want to do anything yourself, use ack, which you can get at http://betterthangrep.com/.