I want to replace new lines in text with coma or space but do not change the last new line.
I know of this question: How to replace newlines with tab characters? - but it does produce an tab on end instead of new line.
So far I have come with:
awk 'NR>1{printf","} {printf $1} END{printf"\n"}'
Is there an easier way to do this? This is not an assignment, I am just curious want to level up my scripting.
This might work for you:
paste -s -d, file
or
paste -s -d" " file
This should do a job:
echo ${$(tr '\n' ',' < file)%?}
or perhaps using sed:
sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/,/g' file
Using perl is not quite as simple as paste, but it does generalize to more cases:
perl -0pe 's/\n(?!$)/,/g' filename # comma
perl -0pe 's/\n(?!$)/ /g' filename # space
The (?!$) is a lookahead assertion that means "not at the end", so the last newline is left alone.
You can use sed command to remove the end ","
# sed -i 's/,$//g' endmodify.txt
# cat endmodify.txt
root
bin
daemon
adm
lp
sync
games
gopher
Thanks.
Related
I have the below lines in a file
Acanthocephala;Palaeacanthocephala;Polymorphida;Polymorphidae;;Profilicollis;Profilicollis_altmani;
Acanthocephala;Eoacanthocephala;Neoechinorhynchida;Neoechinorhynchidae;;;;
Acanthocephala;;;;;;;
Acanthocephala;Palaeacanthocephala;Polymorphida;Polymorphidae;;Polymorphus;;
and I want to remove the repeating semi-colon characters from all lines to look like below (note- there are repeating semi-colons in the middle of some of the above lines too)
Acanthocephala;Palaeacanthocephala;Polymorphida;Polymorphidae;Profilicollis;Profilicollis_altmani;
Acanthocephala;Eoacanthocephala;Neoechinorhynchida;Neoechinorhynchidae;
Acanthocephala;
Acanthocephala;Palaeacanthocephala;Polymorphida;Polymorphidae;Polymorphus;
I would appreciate if someone could kindly share a bash one-liner to accomplish this.
You can use tr with "squeeze":
tr -s ';' < infile
perl -p -e 's/;+/;/g' myfile # writes output to stdout
or
perl -p -i -e 's/;+/;/g' myfile # does an in-place edit
If you want to edit the file itself:
printf "%s\n" 'g/;;/s/;\{2,\}/;/g' w | ed -s foo.txt
If you want to pipe a modified copy of the file to something else and leave the original unchanged:
sed 's/;\{2,\}/;/g' foo.txt | whatever
These replace runs of 2 or more semicolons with single ones.
could be solved easily by substitutions.
I add an awk solution by playing with the FS/OFS variable:
awk -F';+' -v OFS=';' '$1=$1' file
or
awk -F';+' -v OFS=';' '($1=$1)||1' file
Here's a sed version of alaniwi's answer:
sed 's/;\+/;/g' myfile # Write output to stdout
or
sed -i 's/;\+/;/g' myfile # Edit the file in-place
I created a script to add 100 space characters at the end of each line:
#!/bin/ksh
sed -i -e 's/\n/ - 100 spaces - /' $1
but it doesn't work and I think it is because of \n. Any thoughts?
Sed processes the content of the lines without the newline. Your code never sees a newline, so it cannot replace it.
Match the end of the string:
sed -i -e 's/$/ - 100 spaces - /' $1
Although Karoly has already pointed out the error in your script, you could also save yourself typing 100 spaces by using a condition and break to make a sort of loop
sed ':1;/ \{100\}$/!{s/$/ /;b1}' file
Will print 100 space at the end of the line
If there are already spaces at the end and you want to add 100
sed 's/$/1/;:1;/1 \{100\}$/!{s/$/ /;b1};s/1\( *\)$/\1/' file
Just a suggestion to avoid typing 100 spaces (although I'm sure that by this time, you already have!) - use perl:
perl -pe 's/$/" " x 100/e' file
As the other answers have stated, this matches the end of the line and replaces with 100 spaces, using the e modifier to allow you to write " " x 100 and let perl do the work for you.
As with sed, you can modify the file in-place using the -i switch - as with sed, I'd suggest using something like -i.bak to create a backup file file.bak.
Could be the \n as that's undefined by POSIX and so implementation dependent across seds but you also say you want add spaces to the end of a line while your script is trying to replace the end of line with spaces.
In any case this is not a job for sed, just use [s]printf("%100s","") in awk to create a string of 100 blanks, e.g.:
$ echo "foo bar" | awk '{printf "%s%10s%s\n", $1, "", $2}'
foo bar
I am trying to use sed to append the "^A" control character at the end of each line:
sed -i 's/$/^A/' testfile
I want the '\001' special character instead of the "^A" literal string.
Please suggest how to achieve this.
You can use \x01 for ^A i.e.:
sed -i.bak 's/$/\x01/' file
sed -i 's/$/^A/' file
when you press the ^A, just press in this way: Ctrl+(v+a together)
sed 's/$/'$'\01''/' testfile
Explanation
s/$/ — start the sed argument
$'\01' — let bash expand this
'/' — end the sed argument
(I recommend you start off without the rather dangerous -i flag and only add it once you're sure it does what you want.)
Can anyone help me to get substring using sed program?
I have a file with this line:
....
define("BASE", "empty"); # there can be any string (not only "empty").
....
And I need to get "empty" as string variable to my bash script.
At this moment I have:
sed -n '/define(\"BASE\"/p' path/to/file.ext
# returns this line:
# define("BASE", "empty");
# but I need 'empty'
UPD: Thanks to #Jaypal
For now I have bash script:
DBNAME=`sed -n '/define(\"BASE\"/p' path/to/file.ext`
echo $DBNAME | sed -r 's/.*"([a-zA-Z]+)".*/\1/'
It work OK, but if there any way to make the same manipulation with one line of code?
You should use is
sed -n 's/.*\".*\", \"\(.*\)\".*/\1/p' yourFile.txt
which means something (.*) followed by something in quotes (\".*\"), then a comma and a blank space (,), and then again something within quotes (\"\(.*\)\").
The brackets define the part that you later can reuse, i.e. the string within the second quotes. used it with \1.
I put -n front in order to answer the updated question, to get online the line that was manipulated.
This should help -
sed -r 's/.*"([a-zA-Z]+)"\);/\1/' path/to/file.ext
If you are ok with using awk then you can try the following -
awk -F\" '/define\(/{print $(NF-1)}' path/to/file.ext
Update:
DBNAME=$(sed -r '/define\(\"BASE\"/s/.*"([a-zA-Z]+)"\);/\1/' path/to/file.ext)
sed -nr '/^define.*"(.*)".*$/{s//\1/;p}' path/to/file.ext
if your file doesn't change over time (i.e. the line numbers will always be the same) you can take the line, and use delimiters to take your part out:
`sed -n 'Xp' your.file | cut -d ' ' -f 2 |cut -d "\"" -f 2`
assuming X is the line number of your required line
All of the lines with comments in a file begin with #. How can I delete all of the lines (and only those lines) which begin with #? Other lines containing #, but not at the beginning of the line should be ignored.
This can be done with a sed one-liner:
sed '/^#/d'
This says, "find all lines that start with # and delete them, leaving everything else."
I'm a little surprised nobody has suggested the most obvious solution:
grep -v '^#' filename
This solves the problem as stated.
But note that a common convention is for everything from a # to the end of a line to be treated as a comment:
sed 's/#.*$//' filename
though that treats, for example, a # character within a string literal as the beginning of a comment (which may or may not be relevant for your case) (and it leaves empty lines).
A line starting with arbitrary whitespace followed by # might also be treated as a comment:
grep -v '^ *#' filename
if whitespace is only spaces, or
grep -v '^[ ]#' filename
where the two spaces are actually a space followed by a literal tab character (type "control-v tab").
For all these commands, omit the filename argument to read from standard input (e.g., as part of a pipe).
The opposite of Raymond's solution:
sed -n '/^#/!p'
"don't print anything, except for lines that DON'T start with #"
you can directly edit your file with
sed -i '/^#/ d'
If you want also delete comment lines that start with some whitespace use
sed -i '/^\s*#/ d'
Usually, you want to keep the first line of your script, if it is a sha-bang, so sed should not delete lines starting with #!. also it should delete lines, that just contain only a hash but no text. put it all together:
sed -i '/^\s*\(#[^!].*\|#$\)/d'
To be conform with all sed variants you need to add a backup extension to the -i option:
sed -i.bak '/^\s*#/ d' $file
rm -Rf $file.bak
You can use the following for an awk solution -
awk '/^#/ {sub(/#.*/,"");getline;}1' inputfile
This answer builds upon the earlier answer by Keith.
egrep -v "^[[:blank:]]*#" should filter out comment lines.
egrep -v "^[[:blank:]]*(#|$)" should filter out both comments and empty lines, as is frequently useful.
For information about [:blank:] and other character classes, refer to https://en.wikipedia.org/wiki/Regular_expression#Character_classes.
If you want to delete from the file starting with a specific word, then do this:
grep -v '^pattern' currentFileName > newFileName && mv newFileName currentFileName
So we have removed all the lines starting with a pattern, writing the content into a new file, and then copy the content back into the source/current file.
You also might want to remove empty lines as well
sed -E '/(^$|^#)/d' inputfile
Delete all empty lines and also all lines starting with a # after any spaces:
sed -E '/^$|^\s*#/d' inputfile
For example, see the following 3 deleted lines (including just line numbers!):
1. # first comment
2.
3. # second comment
After testing the command above, you can use option -i to edit the input file in place.
Just this!
Here is it with a loop for all files with some extension:
ll -ltr *.filename_extension > list.lst
for i in $(cat list.lst | awk '{ print $8 }') # validate if it is the 8 column on ls
do
echo $i
sed -i '/^#/d' $i
done