I am reading in a list of file names:
*.txt *.xml
which are space delimited. I read this into a variable in my ksh script, and I want to be able to manipulate it before putting each of them into a find command. The problem is, as soon as I do anything with the variable (for instance, breaking it into an array), the * resolves into filenames that are in my script's directory. What I want is for the *.txt to remain unchanged, so I can put that into my find command.
How do I do this? Unfortunately, I'm at work and can't just use perl or some other language.
set -f
turns off globbing in ksh, so * and ? characters are not expanded (globbed).
what's wrong with
'*.txt' '*.xml'
? . Else you have to show us more of your issues. Maybe edit your post to include a small test case that illustrates your problem, plus the desired output or intermediate values.
Related
I am attempting to write a shell script that will take a file name with a wildcard, find all files matching that pattern in current directory, and copy them. My problem is every time I try and use a variable only the first match echo's and thats it.
./copyfiles.ksh cust*.txt
#! /usr/bin/ksh
IN_FILE=${1}
for file in $IN_FILE
do
echo "$file"
done
cust1.txt
This seems to only match the first one even though cust1.txt, cust2.txt, and cust3.txt all exist and when I run it with for file in cust*.txt it works.
The shell expands your argument of "cust*.txt" to a list then passes the list to your script, which then only processes $1 which is cust1.txt.
You want to use $# which will process all arguments passed:
#! /usr/bin/ksh
for file in "$#"
do
echo "$file"
done
I believe there is a limit to how many arguments can be passed this way though. How many files are you having to process? Make sure your version of the shell can handle the number of arguments you are likely to process. If I recall you may need a solution utilizing xargs but I'm a tad rusty to help with that.
In ./copyfiles.ksh cust*.txt the files cust*.txt will be expanded first.
When you do not want to change your copyfiles.ksh script. call it with
./copyfiles.ksh "cust*.txt"
You can also change your script, with something like
IN_FILE="$#" # INFILES would be a better name
I would like to rename my file from myscript.js to .myscript.js. How can I do it with bash? I've tried different options like mv myscript.js \.*, but it doesn't work. I've tried to experiment by adding non-dot symbol like - mv myscript.js m*, but now I don't even know where my file is (of course, I have a copy ;).
You're trying too hard.
mv myscript.js .myscript.js
Brace expansion
mv {,.}myscript.js
You can use rename utility to rename all *.js file by placing a dot before them:
rename 's/(.+\.js)/.$1/' *.js
Or in pure BASH use this for loop:
for i in *.js; do mv "$i" ".$i"; done
Don't use patterns as the target of mv (or cp for that matter). The command won't do what you want or expect, most of the time: Bash will expand the pattern at the moment when you run the command, using the file names as they were before your command was executed. So .* matches nothing (since the file doesn't exist, yet) and m* will match any file or folder which starts with m. Since you didn't get an error, chances are that the last match was a folder.
There is no way to access the previous parameter of the current command. If you want to avoid typing too much, then use Tab for both file names (so you end up with mv myscript.js myscript.js) and then use Ctrl+Left and Ctrl+Right to move quickly to the start of the second argument to insert the missing ..
You can access the parameters of the previous command, though. But that's not a feature of BASH - it's a feature of readline.
I am using Automator on my Mac to set up a service that passes a selected folder to a bash shell script as arguments.
In the script I do:
for f in "$#"; do
printf "%q\n" "$f" | pbcopy
done
if I then do:
echo `pbpaste`
I get the path to my selected folder with spaces escaped (\). I then wanted to use this path to cd into that directory and do a bunch of other stuff (creating a blank directory structure). I hoped I could just do:
cd `pbpaste`
but this doesn't work.
If I type the path manually the cd works so I assume the is some issue with data types or returns or something??
I'll admit I don't really know what this script actually doing and may be going about this all wrong but but if anyone can explain what's going on here and how to get it working it that would be great but even better would be a pointer to a really good resource for a complete beginner to start learning about shell scripting.
I really like the idea of getting into this a bit more but all the resources I have found are either total basics (cd, ls, pwd etc) or really high level and assume a bunch of previous knowledge.
What I'd really like is a full language reference with some actual examples like you find for the languages I am more used to (HTML/CSS/JS/AS3), if such a thing exists.
Cheers for any help :)
I'm agree with #chepner's answer, but for google's results sake, to cd using pbpaste you simply do:
cd $(pbpaste)
When you use the %q format, you are adding literal backslashes to the string, which the shell does not process as escape characters when you use it with cd.
The clipboard is useful for interprocess communication; inside a single script, it's easier to just use variables to hold information temporarily. f already has the path name in it, so just use it:
cd "$f"
Notice I've quoted the expansion of f, so that any spaces in the path name are passed as part of the single argument to cd.
I have a number of files (more than a hundred) that I want to process using Vim. A sample of the files’ contents is as follows:
xyz.csv /home/user/mydocs/abc.txt
/home/user/waves/wav.wav , user_wav.wav
I want this to be replaced by:
xyz.csv /var/lib/mydir/abc.txt
/var/sounds/wav.wav , wav.wav
In each of the files, the changes I need to make are the same. My questions are:
Can I use Vim search and replace functionality by calling it from within a Bash script?
If so, how do I go about it?
P.S. I have searched StackOverflow for similar questions and found some answers using ex scripts, etc. I want to know how I can call an ex script from within a bash script.
While vim is quite powerful, this is not something I would normally use vim for. It can be done using a combination of common command line utilities instead.
I've assumed that the blank line in your example above is actually blank and does not contain spaces or any other whitespace characters. You can use the following to do what you want.
sed -e "s,/home/user/mydocs,/var/lib/mydir," -e "s,/home/user/waves,/var/sounds," -e "/^$/d" file1
You can use that command together with find and a for loop to do this for a bunch of files:
for file in `find . -maxdepth 1 -type f`
do
sed -e "s,/home/user/mydocs,/var/lib/mydir," -e "s,/home/user/waves,/var/sounds," -e "/^$/d" $file
done
In the for loop, the find command above limits the output to all files in the current directory (including dot files), assigning each line from the output of find to the file variable and then running the sed command posted earlier to transform the file the way you want it to be transformed.
This is how you'd invoke an ed script from bash:
ed filename <<END
/^$/d
%s|/home/user/mydocs|/var/lib/mydir|
%s|/home/user/waves|/var/sounds|
%s|, user_|, |
w
q
END
To answer with vim, you can do
vim -e 'bufdo!%s:\(xyz.csv \)/home/user/mydocs/\(abc.txt\n\)\n.*:\1/var/lib/mydir/\2/var/sounds/wav.wav , wav.wav:' -e 'xa' FILES
Note, I had assumed, that the second line is statically replaced, as it had looked like in the question.
If you don't like writing long lines in your script, you can create a file like:
s/FOO/BAR/
" several replacement and other commands
w " write the file
bd " if you want to
Then do:
vim -e "buffdo!source /your_scriptfile" -e "x" FILES
HTH
If all the editing consists in a series of substitutions, the most
idiomatic way of accomplishing it using Vim would be the following.
Open all the target files at once:
vim *.txt
Run the substitution commands on the loaded files:
:argdo %s#/home/user/mydocs#/var/lib/mydir#
:argdo %s#/home/user/waves#/var/sounds#
:argdo %s#, \zsuser_##
...
If changes are correctly made, save the files:
:wall
If the editing you want to automate could not be expressed only
in substitutions, record a macro and run it via the :normal
command:
:argdo norm!#z
(Here z is the name of the macro to be run.)
Lastly, if the editing should be performed from time to time and
needs to be stored in a script, try using the approach described
in the answer to a similar question.
Answer
While most vim users would be aware of the % motion command for executing inclusive commands on the whole document in the current buffer. Most modern versions of vim (ie 6.x+) support actions on regex searches for exclusive actions like so:
:/regex/substitute/match/replace/ # as vim command line
+"/reges/s/match/replace" # as bash cli parameter
This breaks down into vim doing the following,
search for regex and put the cursor at start of found point
call internal substitute function (see :help s or :help substitute) [ could be other commands ]
match string with regex for substitution
replace with new string value
Effectively it operates the same as the :global command.
Notes
Command after regex search can be any command, including '!"shell command"' filter commands.
Reference Help
:help global
:help substitute
:help filter
This is one of my homework exercise.
Write a shell program, which will take a directory as an argument.
The script should then print all the regular files in the directory and all
the recursive directories, with the following information n the given order for
each of the files
File name (full name from the specified directory) file size owner
In case the directory argument is not given, the script should assume the
directory to be the current working directory
I am confused about how to approach this problem. For the listing of files part, I tried ls -R | awk ... but i was not able to do it because I was not able to find a suitable field seperator for awk.
I know its unfair to ask for a solution, but please can you guys give me a hint as how to proceed with the problem? Thanks in advance.
You really don't want to use ls and awk for this. Instead you want to check the documentation for find to figure out what string to put in the following script:
find ${1:-.} -type f -printf "format-string-to-be-determined-by-reader\n"
The problem is that parsing the output of ls is complicated at best and dangerous at worst.
What you'll want to do is use find to produce the list of files and a small shell script to produce the output you want. While there are many possible methods to accomplish this I would use the following general form
while read -r file ; do
# retrieve information about $file
# print that information on one line
done < <(find ...)
With a suitable find command to select the files. To retrieve the metadata about the files I would use stat inside the loop, probably multiple times.
I hope that's enough of a hint, but If you want a complete answer I can provide.
awk is fine.. use " +" as separator.
Bah. Where's the challenge in using ls or find? May as well write a one-liner in perl to do all the work, and then just call the one-liner from a script. ;)
You can do your recursive directory traversal in the shell natively, and use stat to get the size and owner. Basically, you write a function to list the directory (for element in *), and have the function change to the directory and call itself if [[ -d $element ]] is true. Something like
do_print "$elem"
if [[ -d "$elem" ]]
then
cd "$elem"
process_dir
cd ..
fi
or something akin to that.
Yeah, you'll have a zillion system calls to stat, but IMHO that's probably preferable to machine-parsing the output of a program whose output is intended to be human-readable. In this case, where performance is not an issue, it's worth it.
For bonus super happy fun times, change the value of IFS to a value which won't appear in a filename so the shell globbing won't get confused by files containing whitespace in its name. I'd suggest either a newline or a slash.
Or take the easy way out and just use find with printf. :)