I have a trouble with splitting output of cat command (cat /proc/meminfo) into separate lines for working with them.
#!/bin/bash
CURR_DUMP=$(cat /proc/meminfo)
arrIN=(${CURR_DUMP// kB/})
for t in "${arrIN[#]}"
do
echo $t
done
exit 0
But instead of separate lines I have a mess of parts of each line.
What's going wrong with my solution?
Thanks in advance.
You can use mapfile:
mapfile -t arr < <(sed 's/ kB$//' /proc/meminfo)
Related
I have the below lines in a file
Acanthocephala;Palaeacanthocephala;Polymorphida;Polymorphidae;;Profilicollis;Profilicollis_altmani;
Acanthocephala;Eoacanthocephala;Neoechinorhynchida;Neoechinorhynchidae;;;;
Acanthocephala;;;;;;;
Acanthocephala;Palaeacanthocephala;Polymorphida;Polymorphidae;;Polymorphus;;
and I want to remove the repeating semi-colon characters from all lines to look like below (note- there are repeating semi-colons in the middle of some of the above lines too)
Acanthocephala;Palaeacanthocephala;Polymorphida;Polymorphidae;Profilicollis;Profilicollis_altmani;
Acanthocephala;Eoacanthocephala;Neoechinorhynchida;Neoechinorhynchidae;
Acanthocephala;
Acanthocephala;Palaeacanthocephala;Polymorphida;Polymorphidae;Polymorphus;
I would appreciate if someone could kindly share a bash one-liner to accomplish this.
You can use tr with "squeeze":
tr -s ';' < infile
perl -p -e 's/;+/;/g' myfile # writes output to stdout
or
perl -p -i -e 's/;+/;/g' myfile # does an in-place edit
If you want to edit the file itself:
printf "%s\n" 'g/;;/s/;\{2,\}/;/g' w | ed -s foo.txt
If you want to pipe a modified copy of the file to something else and leave the original unchanged:
sed 's/;\{2,\}/;/g' foo.txt | whatever
These replace runs of 2 or more semicolons with single ones.
could be solved easily by substitutions.
I add an awk solution by playing with the FS/OFS variable:
awk -F';+' -v OFS=';' '$1=$1' file
or
awk -F';+' -v OFS=';' '($1=$1)||1' file
Here's a sed version of alaniwi's answer:
sed 's/;\+/;/g' myfile # Write output to stdout
or
sed -i 's/;\+/;/g' myfile # Edit the file in-place
How can remove everything in a pipe delimited file after the second-to-last pipe? Like for the line
David|3456|ACCOUNT|MALFUNCTION|CANON|456
the result should be
David|3456|ACCOUNT|MALFUNCTION
Replace |(string without pipe)|(string without pipe) at the end of each line:
sed 's/|[^|]*|[^|]*$//' inputfile
Using awk, something like
awk -F'|' 'BEGIN{OFS="|"}{NF=NF-2; print}' inputfile
David|3456|ACCOUNT|MALFUNCTION
(or) use cut if you know the number of columns in total, i,e 6 -> 4
cut -d'|' -f -4 inputfile
David|3456|ACCOUNT|MALFUNCTION
The command I would use is
cat input.txt | sed -r 's/(.*)\|.*/\1/' > output.txt
A pure Bash solution:
while IFS= read -r line || [[ -n $line ]] ; do
printf '%s\n' "${line%|*|*}"
done <inputfile
See Reading input files by line using read command in shell scripting skips last line (particularly the answer by Jahid) for details of how the while loop works.
See pattern matching in Bash for information about ${line%|*|*}.
I'm completely lost trying to do something which I thought would be very straightforward : read a file line by line and output everything on one line.
I'm using bash on RHEL.
Consider a simple test case with a file (test.in) with following content:
one
two
three
four
I want to have a script which reads this files and outputs:
one two three four
Done
I tried this (test.sh):
cat test.in | while read in; do
printf "%s " "$in"
done
echo "Done"
The result looks like:
# ./test.sh
foure
Done
#
It seems that the printf causes the cursor to jump to the first position on the same line immediately after the %s. The issues holds when doing echo -e "$in \c".
Any ideas?
another answer:
tr '[:space:]' ' ' < file
echo
This must be safest and most efficient as well. Use \n if you want to only convert new lines instead of any white spaces.
You can use:
echo -- $(<test.in); echo 'Done'
one two three four
Done
echo -- `cat file` | tail -c +4
the -- is to protect you from command line options. But in my shell the -- is printed out. I think that might be a bug. Will have to check.
So you need to check if you have to include | tail -c +4 in your implementation.
I want to replace several strings in several documents. I know how to combine sed commands using the -e option but I don't know how to combine them in a script to run it on multiple files.
I tried to use the 'for' loop but it didn't work.
for i in `ls *.txt`; do sed -e 's/22/twenty two/g' \ -e 's/23/twenty three/g'$i > new/$i; done
Any idea how to do this with shell, awk, Python or Perl?
You are doing:
for i in `ls *.txt`
do
sed -e 's/22/twenty two/g' \ -e 's/23/twenty three/g'$i > new/$i
^ ^^^
why is this here? missing space!
done
Which can be rewritten to:
for i in *.txt # <-- no need to `ls`!!
do
sed -e 's/22/twenty two/g' -e 's/23/twenty three/g' "$i" > new/"$i"
done
So your problems were:
strange \ in between commands.
missing space between sed 's...' and the file name.
And a suggestion:
do not use ls, just expand *.txt.
If you use awk you don't need a loop
awk '{gsub(/22/,"twenty two");gsub(/23/,"twenty three");print > "new/"FILENAME}' *.txt
If you have a lot of substitutions to do you might prefer something like:
awk '
BEGIN {
map[22] = "twenty two"
map[23] = "twenty three"
}
{
for (old in map)
gsub("\\<" old "\\>", map[old])
print > ("new/" FILENAME)
}
' *.txt
The word boundaries (I used GNU awk notation) are important so that 123 doesn't become 1twenty three. That's a problem all the other solutions currently have.
"for" cycles usage may be dangerous when you work with "external" data: to be sure, try to create a .txt file with space in its name.
And yes, you dont need '-e' sed option at all, just combine commands with the simple ";":
while read -r f; do
sed 's/22/twenty two/g; s/23/twenty three/g' "$f" > new/"$f"
done < <(ls *.txt)
You can just use sed "in-place" file modification feature:
while read -r f; do
sed -i 's/22/twenty two/g; s/23/twenty three/g' "$f"
done < <(ls *.txt)
I have a list of files (~1000) and there is 1 file per line in my text file named: 'files.txt'
I have a macro that looks something like the following:
#!/bin/sh
b=$(sed '${1}q;d' files.txt)
cat > MyMacro_${1}.C << +EOF
myFile = new TFile("/MYPATHNAME/$b");
+EOF
and I use this input script by doing
./MakeMacro.sh 1
and later I want to do
./MakeMacro.sh 2
./MakeMacro.sh 3
...etc
So that it reads the n'th line of my files.txt and feeds that string to my created .C macro.
So that it reads the n'th line of my files.txt and feeds that string to my created .C macro.
Given this statement and your tags, I'm going to answer using shell tools and not really address the issue of the .c macro.
The first line of your script contains a sed script. There are numerous ways to get the Nth line from a text file. The simplest might be to use head and tail.
$ head -n "${i}" files.txt | tail -n 1
This takes the first $i lines of files.txt, and shows you the last 1 lines of that set.
$ sed -ne "${i}p" files.txt
This use of sed uses -n to avoid printing by default, then prints the $ith line. For better performance, try:
$ sed -ne "${i}{p;q;}" files.txt
This does the same, but quits after printing the line, so that sed doesn't bother traversing the rest of the file.
$ awk -v i="$i" 'NR==i' files.txt
This passes the shell variable $i into awk, then evaluates an expression that tests whether the number of records processed is the same as that variable. If the expression evaluates true, awk prints the line. For better performance, try:
$ awk -v i="$i" 'NR==i{print;exit}' files.txt
Like the second sed script above, this will quit after printing the line, so as to avoid traversing the rest of the file.
Plenty of ways you could do this by loading the file into an array as well, but those ways would take more memory and perform less well. I'd use one-liners if you can. :)
To take any of these one-liners and put it into your script, you already have the notation:
if expr "$i" : '[0-9][0-9]*$' >/dev/null; then
b=$(sed -ne "${i}{p;q;}" files.txt)
else
echo "ERROR: invalid line number" >&2; exit 1
fi
If I am understanding you correctly, you can do a for loop in bash to call the script multiple times with different arguments.
for i in `seq 1 n`; do ./MakeMacro.sh $i; done
Based on the OP's comment, it seems that he wants to submit the generated files to Condor. You can modify the loop above to include the condor submission.
for i in `seq 1 n`; do ./MakeMacro.sh $i; condor_submit <OutputFile> ; done
i=0
while read file
do
((i++))
cat > MyMacro_${i}.C <<-'EOF'
myFile = new TFile("$file");
EOF
done < files.txt
Beware: you need tab indents on the EOF line.
I'm puzzled about why this is the way you want to do the job. You could have your C++ code read files.txt at runtime and it would likely be more efficient in most ways.
If you want to get the Nth line of files.txt into MyMacro_N.C, then:
{
echo
sed -n -e "${1}{s/.*/myFile = new TFILE(\"&\");/p;q;}" files.txt
echo
} > MyMacro_${1}.C
Good grief. The entire script should just be (untested):
awk -v nr="$1" 'NR==nr{printf "\nmyFile = new TFile(\"/MYPATHNAME/%s\");\n\n",$0 > ("MyMacro_"nr".C")}' files.txt
You can throw in a ;exit before the } if performance is an issue but I doubt if it will be.