Delete CRLF in text file using Bash or Notepad++ - bash

I think this is easier than I think, anyway I would like to know your ideas.
I have this file:
AVP78031.1
AVP78042.1
ATO98108.1
ATO98120.1
But I need to do this:
AVP78031.1
AVP78042.1
ATO98108.1
ATO98120.1
Is there a way in NotePad++ to do this? However, I think this type of edition could do it in Bash Script or even only with the terminal. Is there a way to do this?
If you think that there is another way easier to do this, please let me know.
Any suggestion is always welcome.
Thank you for your time!

Using Notepad++
Ctrl+H
Find what: (\R){2,}
Replace with: $1
CHECK Wrap around
CHECK Regular expression
Replace all
Explanation:
(\R) # group 1, any kind of linebreak
{2,} # may appear 2 or more times
Replacement:
$1 # content of group 1, linebreak
Screenshot (before):
Screenshot (after):

Related

sed command to remove invalid xml character not working

I am really new to regex and I was following other StackOverflow answers to make sed command to remove invalid XML characters.
sed -ie 's/[^\u0009\r\n\u0020-\uD7FF\uE000-\uFFFD\ud800\udc00-\udbff\udfff]//g' myfile.xml
When I run this, it looks like it deletes a bunch of alphabets,,, For example, if it is company, it deletes o,m,p,a,y,etc. Especially lower cases.
There is something wrong with my regex OR maybe it doesn't think it as regex. Would you please help me? Thank you.

replace or delete character '#' in string with bash shell [duplicate]

A console program (translate-shell) has an output with colors and uses special decorate characters for this: ^[[22m, ^[[24m, ^[[1m... and so on.
I'd like to remove them to get a plain text.
I tried with tr -d "^[[22m" and with sed 's/[\^[[22m]//g', but only is removed the number, not the special character ^[
Thanks.
You have multiple options:
https://unix.stackexchange.com/questions/14684/removing-control-chars-including-console-codes-colours-from-script-output
http://www.commandlinefu.com/commands/view/3584/remove-color-codes-special-characters-with-sed
and as -no-ansi as pointed out by Jens in other answer
EDIT
The solution from commandlinefu does the job pretty well:
sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g"
The solution from unix.stackexchange might be better but is much longer and so you would want to create a separate script file because it is so long instead of just doing a shell one-liner.
I found this in the manual about the use of ANSI escape codes:
-no-ansi
Do not use ANSI escape codes.
So you should add this option when starting the program.

How to remove the decorate colors characters in bash output?

A console program (translate-shell) has an output with colors and uses special decorate characters for this: ^[[22m, ^[[24m, ^[[1m... and so on.
I'd like to remove them to get a plain text.
I tried with tr -d "^[[22m" and with sed 's/[\^[[22m]//g', but only is removed the number, not the special character ^[
Thanks.
You have multiple options:
https://unix.stackexchange.com/questions/14684/removing-control-chars-including-console-codes-colours-from-script-output
http://www.commandlinefu.com/commands/view/3584/remove-color-codes-special-characters-with-sed
and as -no-ansi as pointed out by Jens in other answer
EDIT
The solution from commandlinefu does the job pretty well:
sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g"
The solution from unix.stackexchange might be better but is much longer and so you would want to create a separate script file because it is so long instead of just doing a shell one-liner.
I found this in the manual about the use of ANSI escape codes:
-no-ansi
Do not use ANSI escape codes.
So you should add this option when starting the program.

sed: properly replace lines containing paths via bash script

I am trying to write a bash script that will do backup-based DUPLICATE database (Oracle) from Production to Development host. One of the steps is to take a copy of the Production database pfile, edit it properly and then power-up a new instance on another host. I am having difficulties to do the following on my bash script:
Production pfile: (single line taken as an example, all others are the same)
*.audit_file_dest='/u01/app/grid/admin/orcl11/adump'
Let's say that the above line should be changed completely to:
*.audit_file_dest='/u02/another/path'
I've had a look at the examples on StackOverflow, but I really can't understand them. Can someone please help me on that?
Many thanks in advance!
EDIT: Thanks a lot guys. This worked out like a charm. Topic Answered.
If it's exactly as you pasted in the question, you can use sed to substitute:
sed -i "s#audit_file_dest='/u01/app/grid/admin/orcl11/adump'#audit_file_dest='/u02/another/path'#" your_file
if you just need to change the single line in your question, #leafei 's line works for you.
if you want to customize your "pfile", like:
change
nameA='oldA'
nameB='oldB'
nameC='oldC'
to
nameA='newA'
nameB='oldB' #here perhaps you want to keep the old value
nameC='newC'
you could try this:
awk 'BEGIN{s="\x27";FS=OFS="="}{if($1=="nameA"){print $1,s"newA"s;next;} if($1=="nameC"){print $1,s"newC"s;next;}}1' pfile
test:
kent$ echo "nameA='oldA'
nameB='oldB'
nameC='oldC'"|awk 'BEGIN{s="\x27";FS=OFS="="}{if($1=="nameA"){print $1,s"newA"s;next;} if($1=="nameC"){print $1,s"newC"s;next;}}1'
nameA='newA'
nameB='oldB'
nameC='newC'

Search and replace in Shell

I am writing a shell (bash) script and I'm trying to figure out an easy way to accomplish a simple task.
I have some string in a variable.
I don't know if this is relevant, but it can contain spaces, newlines, because actually this string is the content of a whole text file.
I want to replace the last occurence of a certain substring with something else.
Perhaps I could use a regexp for that, but there are two moments that confuse me:
I need to match from the end, not from the start
the substring that I want to scan for is fixed, not variable.
for truncating at the start: ${var#pattern}
truncating at the end ${var%pattern}
${var/pattern/repl} for general replacement
the patterns are 'filename' style expansion, and the last one can be prefixed with # or % to match only at the start or end (respectively)
it's all in the (long) bash manpage. check the "Parameter Expansion" chapter.
amn expression like this
s/match string here$/new string/
should do the trick - s is for sustitute, / break up the command, and the $ is the end of line marker. You can try this in vi to see if it does what you need.
I would look up the man pages for awk or sed.
Javier's answer is shell specific and won't work in all shells.
The sed answers that MrTelly and epochwolf alluded to are incomplete and should look something like this:
MyString="stuff ttto be edittted"
NewString=`echo $MyString | sed -e 's/\(.*\)ttt\(.*\)/\1xxx\2/'`
The reason this works without having to use the $ to mark the end is that the first '.*' is greedy and will attempt to gather up as much as possible while allowing the rest of the regular expression to be true.
This sed command should work fine in any shell context used.
Usually when I get stuck with Sed I use this page,
http://sed.sourceforge.net/sed1line.txt

Resources