I have a tricky question.
I have this .sed script where I need to do some stuff and then execute all the lines of the script with "sed -f script.sed filename.csv"
Inside the script I have used a regexp expression and if when executing the script I use -E, the script works. (sed -E -f script.sed filename.csv)
BUT, the thing is, I CANNOT used it, so in the terminal I need to execute the script like this "sed -f script.sed filename.csv"
Then, the question is, Is there a human possible way to make a regexp expression WORK inside a SCRIPT without using -E in the terminal? Can it be included INSIDE the SCRIPT somehow?
I will write down the script to give more context:
s/ESP/Spain/g #this line changes ESP for Spain in all rows
s/DEN/Denmark/g #this line changes DEN for Denmark in all rows
s/NED/Netherlands/g #this line changes NED for Netherlands in all rows
s/^([^,]*,)([^,]+)((,[^,]*){5},(shooting|judo),)/\1\U\2\L\3/ #This line, changes to UPPER CASE all names of athletes that have as sport either judo or shooting. This is the line that does not work without the -E in the terminal.
/[0],[0],[0],$/d #This line deletes all rows where the last 3 columns have these values
1 c id,name,nationality,sex,date_of_birth,height,weight,sport,gold,silver,bronze,info #this line makes the header not upper case
If you need some data to work with, there you have it:
id,name,nationality,sex,date_of_birth,height,weight,sport,gold,silver,bronze,info
388896171,Abdelraouf Benguit,ALG,male,1985-07-03,1.83,90,judo,0,0,0,
285603057,Abderrahmane Mansouri,ALG,male,1995-01-13,1.72,66,cycling,0,0,0,
545134894,Abderrahmane Meziane,ALG,male,1994-03-07,1.68,62,football,0,0,0,
969824503,Abdullah Alrashidi,IOA,male,1963-08-21,1.83,84,shooting,0,0,1,
897549624,Abdullah Hel Baki,BAN,male,1989-08-01,,,shooting,0,0,0,
153457,Abdullahi Shehu,NGR,male,1993-03-12,1.70,,football,0,0,1,
This is part of a file named athletesv2.csv
Please don't ask why I can't use it, but I'm forbiden from using this option.
BR,
You can use the first three commands as is:
s/ESP/Spain/g #this line changes ESP for Spain in all rows
s/DEN/Denmark/g #this line changes DEN for Denmark in all rows
s/NED/Netherlands/g #this line changes NED for Netherlands in all rows
The fourth command should be "converted" to POSIX BRE:
s/^\([^,]*,\)\([^,]\+\)\(\(,[^,]*\)\{5\},\(shooting\|judo\),\)/\1\U\2\L\3/
Note that this syntax is only supported by GNU sed (the \| as an alternation operator and \+ as one or more quantifier are GNU extensions).
I need to update potentially hundreds of configuration files for a program by adding new lines to text files. I will be adding additional properties, such as background color, to these files and want to automate the process with bash. All of these properties are contained in ".m" files. Its essentially updating the properties of widgets on GUIs. Each object's properties in a gui is labeled with an object type followed by a name. One example of an object is called a Form.
The problem is, each name that follows the object is different so I need to add the line based off of the name of the other properties in each section of the .m file. For example, one file has two form objects. The section for one is called "*FO_mm" while the second object's section is named "*FO_test_area". After the name is an extension for what property is specified, such as "*FO_mm.class:". While the properties each object has tends to vary, I found that all objects share a property called ".parent" so I am using that as a search reference. I want to use the sed command to add a line after the .parent line with the new property, in this case background color. So the idea is to search for a string that starts with "*FO_" and ends with ".parent", with everything inbetween being something different for each section. I want to use a loop to capture the string preceding ".parent" as a variable and attach it to the beginning of the new property line so it matches the current section. Here is my current script:
//The top level directory
script_dir="/project/guis/"
//The extension to look for
file_ext="*.m"
fileList=$(find $script_dir -type f -name "$file_ext")
declare -a file_list
readarray -t file_list < <(printf '%s\n' "$fileList")
cd $script_dir
//Loop through each m file
for m_file in ${file_list[#]}; do
var1=($(grep '*FO_.*.parent:' $m_file))
declare -a var_list
readarray -t var_list < <(printf '%s\n' "$var1")
for i in ${var_list[#]}; do
echo $i
sed -i "/^*FO_.*.parent:.*/a\$i.background: #2b3856 " $m_file
done
done
When I run it, the script adds the line "$i.background: #2b3856" below the .parent line. And the "echo $i" line returns "*FO_mm.parent: FO_mm". So there are several problems.
The value of the variable is not being substituted into the sed statement.
As the echo states, only the first section "*FO_mm" is being saved as a variable, which means the second section "*FO_test_area" is not being implemented.
I only want the object name to be stored and placed into the new line. So the result should give me for the first section "*FO_mm.background: #2b3856" with everything from .background on being tacked on by the last part of the sed statement. Since I am still fairly new to bash and especially sed, I have no idea how to strip the variable down to just the object name.
Here is an example of what a single object section looks like prior to running the script:
*FO_test_area.class: Form
*FO_test_area.static: true
*FO_test_area.parent: FO_mm
*FO_test_area.resizePolicy: "resize_none"
And here is what this section looks like after running the WIP script:
*FO_test_area.class: Form
*FO_test_area.static: true
*FO_test_area.parent: FO_mm
$i.background: #33b342
*FO_test_area.resizePolicy: "resize_none"
Its a lot to describe, but I've hit a wall and I would really appreciate any help you can provide.
If ed is available/acceptable.
The script named script.ed (name it whatever you like).
g/^\*FO_.*\.parent:.*/t.\
s/^\(\*\).*parent: */\1/
s/$/.background: #33b342/
%p
Q
The g/^\*FO_.*\.parent:.*/ will match every line that starts with *FO_ and with .parent: somewhere after it. It will match either *FO_test_area.parent and *FO_mm.class.parent. You gonna have to be specific about the regex to match a specific *FO_.*\.parent: pattern to be able to do a specific search & replace/insert
Here is a specific script for the *FO_test_area.parent:
g/^\*FO_test_area\.parent:.*/t.\
s/^\(\*\).*parent: */\1/
s/$/.background: #33b342/
%p
Q
Modify the script above and add another pattern before the line where %p is at, do the rest of the substitution after that.
Your sample file/data.
*FO_test_area.class: Form
*FO_test_area.static: true
*FO_test_area.parent: FO_mm
*FO_test_area.resizePolicy: "resize_none"
Running the script against your data/file. (file ending with a .m)
ed -s file.m < script.ed
Output
*FO_test_area.class: Form
*FO_test_area.static: true
*FO_test_area.parent: FO_mm
*FO_mm.background: #33b342
*FO_test_area.resizePolicy: "resize_none"
If you're satisfied with the output, next thing is to do the loop.
Doing some adjustment to your script. Instead of a nested for loop, the script is using a while + read loop and Process Substitution. See How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
#!/usr/bin/env bash
script_dir="/project/guis/"
file_ext="*.m"
while IFS= read -r m_file; do
if grep -q '^\*FO_.*.parent:' "$m_file"; then
ed -s "$m_file" < script.ed
fi
done < <(find "$script_dir" -type f -name "$file_ext")
The script.ed is inside the current directory where your script is at. It can be anywhere just need to give it the correct absolute path, e.g.
/path/to/script.ed
If in-place editing is needed change the Q to w inside the ed script.
Remove the line where %p is at if the output is not needed to stdout.
See:
GNU ed
POSIX ed
ed
ed in pdf
MirBSD ed
bash hackers wiki ed
Also your local man pages.
man 1p ed
Please give me suggestion. I need a working solution for sed.exe command line utility (SED build for Windows, version 4.1.5.4013). Or just give me a working UNIX-solution and i'll try to adapt it for my sed (for Win OS).
How to add the new (pretty sophisticated, to be honest) text line after specified line in a text file?
I have input file that contains following text (two lines):
[Pagebar Item Popup Menu]
Feature !Private, Item, MI_IDM_MENU_PAGEBAR_NEW_PAGE = New Page, 1,,,"New item menu"
I need to insert following line between first and second lines (above mentioned ):
Item, "Load tab" = Enable Javascript,,,,"Remove from Start Page.rtl" & Enable plugins & Enable display images & Reload
Is it possible to replace(but not to add new line every time when sed's command running!) an existing first line with two lines (replaced line + the new added string). Or, for an example, to make check if "Item, "Load tab"....." line already exists, avoid undesirable duplicates of the new line which is being inserted?
What I've tried but it didn't work:
sed -i "s/"\[Pagebar Item Popup Menu\]"/"\[Pagebar Item Popup Menu\]"\n"^Item,"Load tab"=Enable Javascript,,,,"Remove from Start Page\.rtl" & Enable plugins & Enable display images & Reload[ 0-9]\{1,\}[.,][0-9]\{3\}$/g" standard_menu.ini
sed -e "/pattern/ {p;s/.*/line_to_insert/;}" input_file > output_file
When pattern matches sed will do the following:
p: print pattern space: in this case this is the matching line
s/.*/line_to_insert/: changes pattern space to line_to_insert
If sed was called without -n then pattern space will be written out
If sed was called with -n then an additional p; command will be needed:
sed -e "/pattern/ {p;s/.*/line_to_insert/;p;}" input_file > output_file
For a unix solution under windows have you considered cygwin?
https://www.cygwin.com/
Secondly, your sed command looks a bit funky - have you tried making it easier on the eyes by using a different set of quotes to surround the expression?
finally, I think you might want cygwin as I don't believe sed alone will do what you want.
How to insert a line in a file between two blocks of known lines (if not already inserted previously), using bash?
This example uses grep as well
Thanks,
//P
I have a bash script that runs and outputs to a text file however the colour codes it uses are also included what i'd like to know is how to remove them from the file, ie
^[[38;1;32mHello^[[39m
^[[38;1;31mUser^[[39m
so I just want to be left with Hello and User, so something like sed -r "special characters" from file A save to file B
sed 's/\^\[\[[^m]*m//g'
remove (all) part of line starting with ^[[ until first m
Some like this:
awk '{sub(/\^\[\[38;1;[0-9][0-9]m/,x);sub(/\^\[\[39m/,x)}1'
Hello
User
i have a template, with a var LINK
and a data file, links.txt, with one url per line
how in bash i can substitute LINK with the content of links.txt?
if i do
#!/bin/bash
LINKS=$(cat links.txt)
sed "s/LINKS/$LINK/g" template.xml
two problem:
$LINKS has the content of links.txt without newline
sed: 1: "s/LINKS/http://test ...": bad flag in substitute command: '/'
sed is not escaping the // in the links.txt file
thanks
Use some better language instead. I'd write a solution for bash + awk... but that's simply too much effort to go into. (See http://www.gnu.org/manual/gawk/gawk.html#Getline_002fVariable_002fFile if you really want to do that)
Just use any language where you don't have to mix control and content text. For example in python:
#!/usr/bin/env python
links = open('links.txt').read()
template = open('template.xml').read()
print template.replace('LINKS', links)
Watch out if you're trying to force sed solution with some other separator - you'll get into the same problems unless you find something disallowed in urls (but are you verifying that?) If you don't, you already have another problem - links can contain < and > and break your xml.
You can do this using ed:
ed template.xml <<EOF
/LINKS/d
.r links.txt
w output.txt
EOF
The first command will go to the line
containing LINKS and delete it.
The second line will insert the
contents of links.txt on the current
line.
The third command will write the file
to output.txt (if you omit output.txt
the edits will be saved to
template.xml).
Try running sed twice. On the first run, replace / with \/. The second run will be the same as what you currently have.
The character following the 's' in the sed command ends up the separator, so you'll want to use a character that is not present in the value of $LINK. For example, you could try a comma:
sed "s,LINKS,${LINK}\n,g" template.xml
Note that I also added a \n to add an additional newline.
Another option is to escape the forward slashes in $LINK, possibly using sed. If you don't have guarantees about the characters in $LINK, this may be safer.