i have a text file that has text.txt
aaa/bbb/ccc/ddd/eee
119
fff/ggg/hhh/iii/jjj
20
now how do i convert this output into 2 columns and store this in another text file
file count
aaa/bbb/ccc/ddd/eee 119
fff/ggg/hhh/iii/jjj 20
i want to do this using shell script
This should work
sed 'N;s/\n/ /' fileName
The above N command is an example sed's multiline capability. N commands takes first_line and second_line and separates them by \n. The pattern is then applied to
first_line\nsecond_line.
In the above example N command is followed by replace of \n with space. As a result the output becomes
first_line second_line
In pure bash:
( echo "file,count"
while read line #Read line by line
do
echo -n "$line," #Print one line, with a comma and without a newline
read line #Get the next line
echo "$line" #Print that line as second column
done < "inputFilename" ) > "outputFilename" #Redirect to output file
should do the trick (assuming you want actual comma separated values).
Related
UPDATED:
Using sed, how can I insert (NOT SUBSTITUTE) a new line on only the first match of keyword for each file.
Currently I have the following but this inserts for every line containing Matched Keyword and I want it to only insert the New Inserted Line for only the first match found in the file:
sed -ie '/Matched Keyword/ i\New Inserted Line' *.*
For example:
Myfile.txt:
Line 1
Line 2
Line 3
This line contains the Matched Keyword and other stuff
Line 4
This line contains the Matched Keyword and other stuff
Line 6
changed to:
Line 1
Line 2
Line 3
New Inserted Line
This line contains the Matched Keyword and other stuff
Line 4
This line contains the Matched Keyword and other stuff
Line 6
You can sort of do this in GNU sed:
sed '0,/Matched Keyword/s//New Inserted Line\n&/'
But it's not portable. Since portability is good, here it is in awk:
awk '/Matched Keyword/ && !x {print "Text line to insert"; x=1} 1' inputFile
Or, if you want to pass a variable to print:
awk -v "var=$var" '/Matched Keyword/ && !x {print var; x=1} 1' inputFile
These both insert the text line before the first occurrence of the keyword, on a line by itself, per your example.
Remember that with both sed and awk, the matched keyword is a regular expression, not just a keyword.
UPDATE:
Since this question is also tagged bash, here's a simple solution that is pure bash and doesn't required sed:
#!/bin/bash
n=0
while read line; do
if [[ "$line" =~ 'Matched Keyword' && $n = 0 ]]; then
echo "New Inserted Line"
n=1
fi
echo "$line"
done
As it stands, this as a pipe. You can easily wrap it in something that acts on files instead.
If you want one with sed*:
sed '0,/Matched Keyword/s//Matched Keyword\nNew Inserted Line/' myfile.txt
*only works with GNU sed
This might work for you:
sed -i -e '/Matched Keyword/{i\New Inserted Line' -e ':a;n;ba}' file
You're nearly there! Just create a loop to read from the Matched Keyword to the end of the file.
After inserting a line, the remainder of the file can be printed out by:
Introducing a loop place holder :a (here a is an arbitrary name).
Print the current line and fetch the next into the pattern space with the ncommand.
Redirect control back using the ba command which is essentially a goto to the a place holder. The end-of-file condition is naturally taken care of by the n command which terminates any further sed commands if it tries to read passed the end-of-file.
With a little help from bash, a true one liner can be achieved:
sed $'/Matched Keyword/{iNew Inserted Line\n:a;n;ba}' file
Alternative:
sed 'x;/./{x;b};x;/Matched Keyword/h;//iNew Inserted Line' file
This uses the Matched Keyword as a flag in the hold space and once it has been set any processing is curtailed by bailing out immediately.
If you want to append a line after first match only, use AWK instead of SED as below
awk '{print} /Matched Keyword/ && !n {print "New Inserted Line"; n++}' myfile.txt
Output:
Line 1
Line 2
Line 3
This line contains the Matched Keyword and other stuff
New Inserted Line
Line 4
This line contains the Matched Keyword and other stuff
Line 6
I want to replace some characters in header and footer of a file. If say, I want to replace 5th to 9th character how do I do it? I need to use bash or a shell command.
I want to do something like this
s="abcdabcd"
s=s=s[0]+"12"+s[4:]
>a12dabcd
I have a string of exact length I can substitute and the start and end of replacement. I want to put the generated replacement back into the file.
Example:
I have this header:
HEADER 22aabbccdd23aabbccdd
I get these start and end indices : 2,10
I get this string: xyz56789
I want this: HEADER 22xyz5678923aabbccdd
to replace the existing 1st line in the file.
This can be done with Perl:
perl -i -lpe 'if ($. == 1 || eof) { substr($_, 1, 2) = "12" }' input.txt
-i: modify file in place
-l: automatically strip newlines from input and add them back on output
-p: iterate over lines of the input file and print them back out
-e CODE: what to do for each line
First we check whether the current line number ($.) is 1 (i.e. we're processing the first line of the file) or we have reached the end of the file (i.e. the line currently being processed is the last line of the file). If the condition is true, we take the substring of the current line ($_) starting from offset 1 of length 2 and set it to "12".
Simply with sed:
input.txt:
$ cat input.txt
22aabbccdd23aabbccdd
asasdfsdfd234234234234
$ sed -Ei '1 s/(..).{8}/\1xyz56789/' input.txt
Result:
22xyz5678923aabbccdd
asasdfsdfd234234234234
How can I print a command output like one from rm -rv * in a single line ? I think it would need \r but I can't figure out how.
I would need to have something like this :
From:
removed /path/file1
removed /path/file2
removed /path/file3
To : Line 1 : removed /path/file1
Then : Line 1 : removed /path/file2
Then : Line 1 : removed /path/file3
EDIT : I may have been misunderstood, I want to have the whole process beeing printing in a single same line, changing as the command outputs an another line (like removed /path/file123)
EDIT2 : The output is sometimes too long to be display in on line (very long path). I would need something that considers that problem too :
/very/very/very/long/path/to/a/very/very/very/far/file/with-a-very-very-very-long-name1
/very/very/very/long/path/to/a/very/very/very/far/file/with-a-very-very-very-long-name2
/very/very/very/long/path/to/a/very/very/very/far/file/with-a-very-very-very-long-name3
Here's a helper function:
shopt -s checkwinsize # ensure that COLUMNS is available w/ window size
oneline() {
local ws
while IFS= read -r line; do
if (( ${#line} >= COLUMNS )); then
# Moving cursor back to the front of the line so user input doesn't force wrapping
printf '\r%s\r' "${line:0:$COLUMNS}"
else
ws=$(( COLUMNS - ${#line} ))
# by writing each line twice, we move the cursor back to position
# thus: LF, content, whitespace, LF, content
printf '\r%s%*s\r%s' "$line" "$ws" " " "$line"
fi
done
echo
}
Used as follows:
rm -rv -- * 2>&1 | oneline
To test this a bit more safely, one might use:
for f in 'first line' 'second line' '3rd line'; do echo "$f"; sleep 1; done | oneline
...you'll see that that test displays first line for a second, then second line for a second, then 3rd line for a second.
If you want a "status line" result that is showing the last line output by the program where the line gets over-written by the next line when it comes out you can send the output for the command through a short shell while loop like this:
YourCommand | while read line ; do echo -n "$line"$' ...[lots of spaces]... \r' ; done
The [Lots of spaces] is needed in case a shorter line comes after a longer line. The short line needs to overwrite the text from the longer line or you will see residual characters from the long line.
The echo -n $' ... \r' sends a literal carriage return without a line-feed to the screen which moves the position back to the front of the line but doesn't move down a line.
If you want the text from your command to just be output in 1 long line, then
pipe the output of any command through this sed command and it should replace the carriage returns with spaces. This will put the output all on one line. You could change the space to another delimiter if desired.
your command | sed ':rep; {N;}; s/\n/ /; {t rep};'
:rep; is a non-command that marks where to go to in the {t rep} command.
{N;} will join the current line to the next line.
It doesn't remove the carriage return but just puts the 2 lines in the buffer to be used for following commands.
s/\n/ /; Says to replace the carriage return character with a space character. They space is between the second and third/ characters.
You may need to replace \r\n depending on if the file has line feeds. UNIX files don't unless they came from a pc and haven't been converted.
{t rep}; says that if the match was found in the s/// command then go to the :rep; marker.
This will keep joining lines, removing the \n, then jumping to :rep; until there are no more likes to join.
I want to extract the first column of the last line of a text file. Instead of output the content of interest in another file and read it in again, can I just use some command to read it into a variable directly?
For exampole, if my file is like this:
...
123 456 789(this is the last line)
What I want is to read 123 into a variable in my shell script. How can I do that?
One approach is to extract the line you want, read its columns into an array, and emit the array element you want.
For the last line:
#!/bin/bash
# ^^^^- not /bin/sh, to enable arrays and process substitution
read -r -a columns < <(tail -n 1 "$filename") # put last line's columns into an array
echo "${columns[0]}" # emit the first column
Alternately, awk is an appropriate tool for the job:
line=2
column=1
var=$(awk -v line="$line" -v col="$column" 'NR == line { print $col }' <"$filename")
echo "Extracted the value: $var"
That said, if you're looking for a line close to the start of a file, it's often faster (in a runtime-performance sense) and easier to stick to shell builtins. For instance, to take the third column of the second line of a file:
{
read -r _ # throw away first line
read -r _ _ value _ # extract third value of second line
} <"$filename"
This works by using _s as placeholders for values you don't want to read.
I guess with "first column", you mean "first word", do you?
If it is guaranteed, that the last line doesn't start with a space, you can do
tail -n 1 YOUR_FILE | cut -d ' ' -f 1
You could also use sed:
$> var=$(sed -nr '$s/(^[^ ]*).*/\1/p' "file.txt")
The -nr tells sed to not output data by default (-n) and use extended regular expressions (-r to avoid needing to escape the paranthesis otherwise you have to write \( \))). The $ is an address that specifies the last line. The regular expression anchors the beginning of the line with the first ^, then matches everything that is not a space [^ ]* and puts that the result into a capture group ( ) and then gets rid of the rest of the line .* by replacing the line with the capture group \1, then print p to print the line.
I have a large file of a bilingual lexicon with lines formatted as:
abatement: disminucion; mitigacion; moderacion; rebaja; deduccion; supresion; anulacion
I'm trying to find out which line has the most translated words, and so am looking to find the line with the most occurrences of ;, and then echo the English word.
I've managed to get something close but it uses sed to trim off data, meaning I can't match the English word back to the line.
Any ideas?
awk -F'[:;]' '{if(NF>n){n=NF;w=$1}}END{print w}' filename
Treating ; as a field separator, the line with the ; will have the most fields.
while IFS=';' read -a fields; do
n=${#fields[#]}
if (( n > max )); then
max=$n
english=${fields[0]%:}
fi
done < file.txt
echo "$english"