I am a beginer in Shell script. Below is my requirement in UNIX Korn Shell.
Example:
When we list files using ls command redirect to a file the file names will be stored as below.
$ ls FILE*>FLIST.TXT
$ cat FLIST.TXT
FILE1
FILE2
FILE3
But I need output as below with a prefixed constant string STR,:
$ cat FLIST.TXT
STR,FILE1
STR,FILE2
STR,FILE3
Please let me what should be the ls command to acheive this output.
You can't use ls alone to append data before each file. ls exists to list files.
You will need to use other tools along side ls.
You can append to the front of each line using the sed command:
cat FLIST.TXT | sed 's/^/STR,/'
This will send the changes to stdout.
If you'd like to change the actual file, run sed in place:
sed -i -e 's/^/STR,/' FLIST.TXT
To do the append before writing to the file, pipe ls into sed:
ls FILE* | sed 's/^/STR,/' > FLIST.TXT
The following should work:
ls FILE* | xargs -i echo "STR,{}" > FLIST.TXT
It takes every one of the file names filtered by ls and adds the "STR," prefix to it prior to the appending
Related
We have a MarkDown file where we are storing versions of multiple components.
Below is the sample .md file
component1=1.2
components2=2.3
component3=`cat file1 | grep 'App_Version' | grep -P '(?<==).*' && rm -rf file1`
Here the component3 version is dynamic, so we are executing the command to get the version.
Need help on accomplishing this in correct way.
Markdown is not a scripting language, so you probably need one form or another of preprocessing. Example with GNU m4 (but any preprocessor with similar capabilities would do the job):
$ cat sample.m4
m4_changequote(`"""', `"""')m4_dnl
component1=1.2
components2=2.3
component3=m4_esyscmd("""grep -Po '(?<=App_Version=).*' file1 && rm -f file1""")m4_dnl
component4=foo
$ cat file1
App_Version=4.0.2
$ m4 -P sample.m4 > sample.md
$ cat sample.md
component1=1.2
components2=2.3
component3=4.0.2
component4=foo
$ ls file1
ls: cannot access 'file1': No such file or directory
Explanations:
The -P option of m4 modifies all builtin macro names so they all start with the m4_ prefix. It is not absolutely needed but it makes the source code easier to read.
The sample.m4 file is your source file, the one you edit. The:
m4 -P sample.m4 > sample.md
command preprocesses the source file to produce the markdown file.
The m4_changequote macro at the beginning of sample.m4 changes the quotes that m4 uses for text strings. Use any left and right quotes you want (""" in our example) as long as it is not used in your markdown text.
m4_dnl is the macro that suppresses the rest of the line, including the newline character.
m4_esyscmd("""cmd""") substitutes the output of the cmd shell script.
Note: I assumed that you wanted grep -Po '(?<=App_Version=).*' file1 instead of cat file1 | grep 'App_Version' | grep -P '(?<==).*' which looks like several anti-patterns at once.
script.sh takes two files as arguments, and it calls a python script that opens them and does further processing on these files:
script.sh file1.txt file2.txt
I would like to remove all lines from file1.txt that start with #.
grep -v '^#' file1.txt
I tried the following but it fails :
script.sh $(grep -v '^#' file1.txt) file2.txt
How can I pipe the output of grep as the first file argument of script.sh?
script.sh <(grep -v '^#' file1.txt) file2.txt
Command substitution as you had it, $(), will insert the output from the grep command verbatim as arguments to script.sh. In contrast, to quote the bash man page:
Process substitution allows a process's input or output to be referred to using a filename. It takes the form of <(list) or >(list). The process list is run asynchronously, and its input or output appears as a filename. This filename is passed as an argument to the current command as the result of the expansion.
and
If the <(list) form is used, the file passed as an argument should be read to obtain the output of list.
So, my understanding is that when you use $(), the output of your grep command is substituted into the final statement with a result something like:
script.sh contents of file1 without comments \n oops newline etc file2.txt
But when you use <(), the grep command is run and the output goes into a named pipe or temporary file or some equivalent, and the name of that "file" is substituted into your statement, so you end up with something along the lines of
script.sh /dev/fd/temp_thing_containing_grep_results file2.txt
I'm writing a script
One goal of the script is to append a new codeline after a match, with foo, in multiple .yml files recursively.
# Append new line after match in multiple files with sed
## sed -i -se "s/\foo/bar/g" *.yml
grep -rl foo * .| xargs sed -i -e "s/\foo/a bar/g" *.yml
I expected after every match with foo, because of /a, bar would be added on a new line in all .yml files.
I get unexpected Sed outpout:
sed: can't read *.yml: No such file or directory
Details:
OS: Debian GNU/Linux 10
sed --version: sed (GNU sed) 4.7
You're supplying the filename arguments with xargs. You shouldn't also use *.yml as an argument. The shell will expand that to the .yml files in the current directory, and you'll get an error if there aren't any.
If you want grep -r to only examine .yml files, use the --include option.
grep -rl --include='*.yml' foo . | xargs sed -i -e "s/foo/a bar/g"
Note that the s command isn't used for appending a new line after a line. It's used for substituting within the same line. s/foo/ a bar/g replaces all the foo on the line with a bar. If you want to append, get rid of the s command. Also, you don't put /g at the end of the a command.
grep -rl --include='*.yml' foo . | xargs sed -i -e "/foo/a bar"
See How to insert text after a certain string in a file?
I have a file that contains a list of filenames on each line.
myfile.txt:
somepath/Documents/a.txt
somepath/Documents/b.txt
somepath/Documents/c.txt
This file can contain any number of lines. What I want to do is then run a command that runs cat with each line being an input, such as:
cat <line 1> <line 2> <line 3> new_combination_file.txt
I was looking this up and I think I should be using xargs to do this, but when I looked up examples and the man page, it didn't make sense to me. Can someone help?
Use a while read loop to read each line into a variable
while read -r filename
do
cat "$filename"
done < myfile.txt > new_combination_file.txt
You can also use xargs:
xargs cat < myfile.txt > new_combination_file.txt
However, this won't work if any filenames contain spaces.
You can use xargs to do this like so:
xargs -t -a myfile.txt cat >new_combination_file.txt
If you want to use xargs command, just use
cat myfile.txt | xargs -I {} cat {}
-I specifies that the following command will be executed for each line of the input (that is, cat myfile.txt in this example) and
{} represents the value of each line
If you want to combine the output produced just use the regular redirect > command followed by the filename.
Hope this helps.
I have file File1.txt in which i have to replace a text using sed command
File1.txt contents
EURAMOUNTTOBEREPLACED
I have a AIX shell script for replacing the text AMOUNTTOBEREPLACED
Contents of the shell script
sum=27
sed 's/AMOUNTTOBEREPLACED/"$sum"/g' File1.txt >> temp
mv temp FileNew.txt
After executing the script, the contents of FileNew.txt is as below
EUR"$sum"
But Expected output should be
EUR27
Please help how to do?
I think the one you want is like this:
sed 's/AMOUNTTOBEREPLACED/'$sum'/g' File1.txt >> temp
Basically single qouting takes the string to sed and skips shell which is wrong. You want the shell to interpret the variable so thats what i did. And further if you happen to have gnu version of sed. Then you can do
sed -i 's/AMOUNTTOBEREPLACED/'$sum'/g' File1.txt
which compressed these two statement in your code to one in above:
sed 's/AMOUNTTOBEREPLACED/'$sum'/g' File1.txt >> temp
mv temp FileNew.txt
use the below sed command instead:
sed -e "s/AMOUNTTOBEREPLACED/\"${sum}\"/g"