I have a script one of my Professors from college gave us to modify however I cannot seem to figure out exactly what it does. Can anyone help me to understand it a bit better?
for i
do grep look $i; done
From what I can gather it greps the value of the variable i, which could be a file or directory. However I am not familiar with how the look command comes into play. I would greatly appreciate any tips you could offer.
look isn't a command, it's the first parameter to grep. So it will search for the word look in the file named $i. (grep will not search folders unless you pass in -R as in grep -R look $i.)
The confusing bit is that for i usually comes with an in WORDS specified, so for i in one two three will run the commands between do and done three times: once with variable i = "one", once with i = "two", and once with i = "three". However, the bash manual explains what to do if in isn't specified:
If ‘in words’ is not present, the for command executes the commands once for each positional parameter that is set, as if ‘in "$#"’ had been specified [...].
So, in short, if your script is in a file named foo.sh, then calling foo.sh file1 file2 will look for the word look in files "file1" and "file2".
Related
i am using opensuse as a virtual machine on my laptop.
this question is about code that i need to do for my homework.
I need to make a script with a variable that shows the amount of entries in a directory.
when i write the exact command in a bash script the output is diffrent from when i run it directly from the CLI
#! /bin/bash
clear
ENTRIES=$(ls /tmp | wc -l)
echo "the amount of entries is" "$ENTRIES"
When i run this script the output will be 53
but when i type the command "ls /tmp | wc -l" in the terminal/CLI i get 61
does anyone know how to solve/explain this?
I got confused and went to look online for answers but I could not find any that's why I am asking this question
thanks for the effort
Sorry for any spelling mistakes. I‘m from the Netherlands.
The command substitution might be implemented in a way which creates a temporary file.
More likely, the number of files in /tmp naturally varies over time, and you postulate a causation where there just happened to be a correlation.
A better way to implement this avoids parsing ls output using either an array
#!/bin/bash
tmpfiles=(/tmp/*)
echo "$(#tmpfiles[#]} files in /tmp"
or just enumerating the files, which is portable to POSIX sh:
#!/bin/sh
set -- /tmp/*
echo "$# files in /tmp"
Printing out the array or list of arguments should reveal which files exactly were present.
As an aside, don't use upper case for your private variables; uppercase variable names are reserved for system variables.
Starting with Xcode 10, build script phases can use file lists (.xcfilelist) for input and output instead of specifying input/output files directly. Those files seem to support comments (the WWDC sample showed command line comments on top), blank lines (also in the sample), and otherwise expect one file path per line. If these file contain build settings (e.g. $(SRCROOT)), these are expanded prior to calling the script, just like they would have been expanded if the file path was directly given as input/output file.
This sounds like a great feature but how would you use these file lists in your actual script?
When specifying the files directly, you had the shell variables SCRIPT_INPUT_FILE_COUNT and SCRIPT_OUTPUT_FILE_COUNT and then one variable for each input/output file, named SCRIPT_INPUT_FILE_# and SCRIPT_OUTPUT_FILE_# where # used to be an up-counting number. Assuming that you have an equal number of input/output file, this script would print them all:
#!/bin/sh
: $((i=0))
while [ $i -lt "$SCRIPT_INPUT_FILE_COUNT" ]
do
eval fileIn="\$SCRIPT_INPUT_FILE_${i}"
eval fileOut="\$SCRIPT_OUTPUT_FILE_${i}"
echo "$fileIn --> $fileOut"
: $((i=i+1))
done
This is a clean POSIX compatible shell script, yes, you can make it even nicer when requiring bash but the code above should work with every sh compatible shell (which it also promisses when using #!/bin/sh and not #!/bin/bash).
But when using file lists, SCRIPT_INPUT_FILE_COUNT is 0. Instead you get SCRIPT_INPUT_FILE_LIST_COUNT and SCRIPT_OUTPUT_FILE_LIST_COUNT, and the variables SCRIPT_INPUT_FILE_LIST_# and SCRIPT_OUTPUT_FILE_LIST_#, containing the paths to the pre-processed file lists, where all comments and blank lines have been stripped and all build settings have already been expanded.
Now, how would I go about using these file lists in my script? How would the tiny sample script above produce the same output using file lists in Xcode? I'm not really good at shell scripting and I'm looking for a clean solution that doesn't require any other script interpreter but sh.
This will dynamically construct the SCRIPT_INPUT_FILE_LIST_0, SCRIPT_INPUT_FILE_LIST_1, etc. values and access them from the environment vars passed to the script by Xcode. Swap out the echo "${file_path}" line if you want to do something other than printing each of the lines from the xcfilelist(s).
#!/usr/bin/env bash
for index in $(seq $SCRIPT_INPUT_FILE_LIST_COUNT); do
# 1 => `SCRIPT_INPUT_FILE_LIST_0`
filelist=SCRIPT_INPUT_FILE_LIST_$((index-1))
# `SCRIPT_INPUT_FILE_LIST_0` => value in $SCRIPT_INPUT_FILE_LIST_0
filelist_path=${!filelist}
while read -r file_path; do
echo "${file_path}"
done <$filelist_path
done
I'm not aware of a way to get access to the file list itself inside the shellscript. However, the idea of a file list is you ideally have one for however many files there are. So, I usually hardcode it to the same value I gave xcode. It's a bit duplicated, but not a whole lot:
set -e
while read file; do
EXPANDED=`eval echo "$file"`
echo "do something with $EXPANDED"
done <"${SRCROOT}/path/to/files.xcfileslist"
As a side effect of this read device we strip whitespace and do some other light processing. If you are particular about how you want this to happen, see this SO answer.
I believe most (all?) build settings are exported into the script as environment variables. So by evaluating them with eval here we expand them.
Use of eval opens up the possibility that a malicious file list can execute code. Then again it's probably located in the same place as the build script you're executing so I'm not sure it's a very practical problem. Other shells have more secure ways of going about this, but I'm not aware of any for vanilla sh and default macOS.
I am attempting to write a shell script that will take a file name with a wildcard, find all files matching that pattern in current directory, and copy them. My problem is every time I try and use a variable only the first match echo's and thats it.
./copyfiles.ksh cust*.txt
#! /usr/bin/ksh
IN_FILE=${1}
for file in $IN_FILE
do
echo "$file"
done
cust1.txt
This seems to only match the first one even though cust1.txt, cust2.txt, and cust3.txt all exist and when I run it with for file in cust*.txt it works.
The shell expands your argument of "cust*.txt" to a list then passes the list to your script, which then only processes $1 which is cust1.txt.
You want to use $# which will process all arguments passed:
#! /usr/bin/ksh
for file in "$#"
do
echo "$file"
done
I believe there is a limit to how many arguments can be passed this way though. How many files are you having to process? Make sure your version of the shell can handle the number of arguments you are likely to process. If I recall you may need a solution utilizing xargs but I'm a tad rusty to help with that.
In ./copyfiles.ksh cust*.txt the files cust*.txt will be expanded first.
When you do not want to change your copyfiles.ksh script. call it with
./copyfiles.ksh "cust*.txt"
You can also change your script, with something like
IN_FILE="$#" # INFILES would be a better name
I use ZSH for my terminal shell, and whilst I've written several functions to automate specific tasks, I've never really attempted anything that requires the functionality I'm after at the moment.
I've recently re-written a blog using Jekyll and I want to automate the production of blog posts and finally the uploading of the newly produced files to my server using something like scp.
I'm slightly confused about the variable bindings/usage in ZSH; for example:
DATE= date +'20%y-%m-%d'
echo $DATE
correctly outputs 2011-08-23 as I'd expect.
But when I try:
DATE= date +'20%y-%m-%d'
FILE= "~/path/to/_posts/$DATE-$1.markdown"
echo $FILE
It outputs:
2011-08-23
blog.sh: line 4: ~/path/to/_posts/-.markdown: No such file or directory
And when run with what I'd be wanting the blog title to be (ignoring the fact the string needs to be manipulated to make it more url friendly and that the route path/to doesn't exist)
i.e. blog "blog title", outputs:
2011-08-23
blog.sh: line 4: ~/path/to/_posts/-blog title.markdown: No such file or directory
Why is $DATE printing above the call to print $FILE rather than the string being included in $FILE?
Two things are going wrong here.
Firstly, your first snippet is not doing what I think you think it is. Try removing the second line, the echo. It still prints the date, right? Because this:
DATE= date +'20%y-%m-%d'
Is not a variable assignment - it's an invocation of date with an auxiliary environment variable (the general syntax is VAR_NAME=VAR_VALUE COMMAND). You mean this:
DATE=$(date +'20%y-%m-%d')
Your second snippet will still fail, but differently. Again, you're using the invoke-with-environment syntax instead of assignment. You mean:
# note the lack of a space after the equals sign
FILE="~/path/to/_posts/$DATE-$1.markdown"
I think that should do the trick.
Disclaimer
While I know bash very well, I only started using zsh recently; there may be zshisms at work here that I'm not aware of.
Learn about what a shell calls 'expansion'. There are several kinds, performed in a particular order:
The order of word expansion is as follows:
tilde expansion
parameter expansion
command substitution
arithmetic expansion
pathname expansion, unless set -f is in effect
quote removal, always performed last
Note that tilde expansion is only performed when the tilde is not quoted; viz.:
$ FILE="~/.zshrc"
$ echo $FILE
~/.zshrc
$ FILE=~./zshrc
$ echo $FILE
/home/user42/.zshrc
And there must be no spaces around the = in variable assignments.
Since you asked in a comment where to learn shell programming, there are several options:
Read the shell's manual page man zsh
Read the specification of the POSIX shell, http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html, especially if you want to run your scripts on different operating systems (and you will find yourself in that situation one fine day!)
Read books about shell programming.
Hang out in the usenet newsgroup comp.unix.shell where a lot of shell wizards answer questions
This is one of my homework exercise.
Write a shell program, which will take a directory as an argument.
The script should then print all the regular files in the directory and all
the recursive directories, with the following information n the given order for
each of the files
File name (full name from the specified directory) file size owner
In case the directory argument is not given, the script should assume the
directory to be the current working directory
I am confused about how to approach this problem. For the listing of files part, I tried ls -R | awk ... but i was not able to do it because I was not able to find a suitable field seperator for awk.
I know its unfair to ask for a solution, but please can you guys give me a hint as how to proceed with the problem? Thanks in advance.
You really don't want to use ls and awk for this. Instead you want to check the documentation for find to figure out what string to put in the following script:
find ${1:-.} -type f -printf "format-string-to-be-determined-by-reader\n"
The problem is that parsing the output of ls is complicated at best and dangerous at worst.
What you'll want to do is use find to produce the list of files and a small shell script to produce the output you want. While there are many possible methods to accomplish this I would use the following general form
while read -r file ; do
# retrieve information about $file
# print that information on one line
done < <(find ...)
With a suitable find command to select the files. To retrieve the metadata about the files I would use stat inside the loop, probably multiple times.
I hope that's enough of a hint, but If you want a complete answer I can provide.
awk is fine.. use " +" as separator.
Bah. Where's the challenge in using ls or find? May as well write a one-liner in perl to do all the work, and then just call the one-liner from a script. ;)
You can do your recursive directory traversal in the shell natively, and use stat to get the size and owner. Basically, you write a function to list the directory (for element in *), and have the function change to the directory and call itself if [[ -d $element ]] is true. Something like
do_print "$elem"
if [[ -d "$elem" ]]
then
cd "$elem"
process_dir
cd ..
fi
or something akin to that.
Yeah, you'll have a zillion system calls to stat, but IMHO that's probably preferable to machine-parsing the output of a program whose output is intended to be human-readable. In this case, where performance is not an issue, it's worth it.
For bonus super happy fun times, change the value of IFS to a value which won't appear in a filename so the shell globbing won't get confused by files containing whitespace in its name. I'd suggest either a newline or a slash.
Or take the easy way out and just use find with printf. :)