I have three variables I write into a text file. For now it is into a variable, because try and error is faster like this. These three variables are produced by the script before and each variable has a column with values in them. Variable one for example looks like this:
hour
minute
minute
day
I put them together using this code:
New_fileloc=$(paste <(echo "$Grabinterval") <(echo "$Filelocation") <(echo "$Time") --delimiters ';' | sed -e 's/^\|$/"/g' -e 's/\;/";"/')
At the end I need each line in each column in double quotes and separated by a semicolon. At the moment I am doing that using sed in that one liner. And that works mostly fine. My output looks like this:
"hour";"A_path/to/somewhere/;2016-02-10 17:07:00Z"
"minute";"A_path/to/somewhere/;2016-01-29 17:26:20Z"
"minute";"A_path/to/somewhere/;2016-01-29 17:26:20Z"
"day";"A_path/to/somewhere/;2016-01-29 00:07:00Z"
The first semicolon gets replaces with ";", but the second one in each line does not. I have no idea why.
In your last command to sed you are missing a g
s/\;/";"/g
Your original command with 's/\;/";"/' will only make one replacement, the first.
Related
I have a file with n number of lines which look like this:
a,b,c,,,,d
a,b,,,,c,d
a,,,,b,c,d
what I want to do is to delete the repeated commas from each line if there are repeated commas within that line. Therefore, the lines from my file should look like this:
a,b,c,d
a,b,c,d
a,b,c,d
I was trying to use grep or awk but I think I'm not really understanding those commands. I am new at bash and I'm kinda stuck so I would really appreciate your help!!
sed is probably the tool for this. Something like:
sed 's/,,*/,/g'
Which you can use with pipes in many ways.
The g option is global ( not m=multiple ) indicating that the phrase can occur more than once on the line - otherwise only the first on each line is modified.
Soon someone will come along and refer us to a duplicate. That's OK.
this will be shorter
$ tr -s , <file
a,b,c,d
a,b,c,d
a,b,c,d
Use this Perl one-liner to replace 1 or more occurrence of a comma with exactly 1 occurrence, multiple matches per line:
perl -pe 's/,+/,/g' in_file > out_file
The Perl one-liner uses these command line flags:
-e : Tells Perl to look for code in-line, instead of in a file.
-p : Loop over the input one line at a time, assigning it to $_ by default. Add print $_ after each loop iteration.
I’m having the following issue:
I have the csv file with data inside looks like this:
“1,””name””,””surname””,””age””,””city”””
“2,””Peter””,””Parker””,””30””,””NY”””
“3,””marry””,””Jane””,””30””,””NY”””
Is it possible using bash to delete the first and the last double quote from each row and then first and last double quote from each field in it?
To get something like this:
1,”name”,”surname”,”age”,”NY”
3,”marry”,”Jane”,”30”,”NY”
I would be grateful for some hints. Thanks
To get you started:
echo '"1,""name"",""surname"",""age"",""city"""' | sed "s/\"\"/\"/g" | sed 's/^\"\(.*\)\"$/\1/'
OUPUT
You can take that, adjust it to run over a file, line by line (instead of the first echo and output into another file
Presuming your input looks like this:
"1,""name"",""surname"",""age"",""city"""
"2,""Peter"",""Parker"",""30"",""NY"""
"3,""marry"",""Jane"",""30"",""NY"""
Note the actual "'s not the ”” in your code:
You can then sed multiple things and chain them together e.g.
sed -e "s/\"\"\"/\"/g" -e "s/\"\"/\"/g" input.txt
This first replaces the triple quotes """, reducing them to double quotes "" and then reduces them further.
Final output:
"1,"name","surname","age","city"
"2,"Peter","Parker","30","NY"
"3,"marry","Jane","30","NY"
If you have special characters then simply replace them in the code e.g.:
$ cat input.txt
“1,””name””,””surname””,””age””,””city”””
“2,””Peter””,””Parker””,””30””,””NY”””
“3,””marry””,””Jane””,””30””,””NY”””
$ sed -e "s/\”\”\”/\”/g" -e "s/\”\”/\”/g" input.txt
“1,”name”,”surname”,”age”,”city”
“2,”Peter”,”Parker”,”30”,”NY”
“3,”marry”,”Jane”,”30”,”NY”
Though I think this input is a transpose error in your question.
Using sed:
sed 's/^"\(.*\)"$/\1/;s/"\+/"/g' file
The first substitution removes the outer double quote on the whole line.
The second substitution replaces the parameter quote to only one double quote.
I want the string index.xml to be appended when I see something ending in /feed/ and starting with http://a.b.c
Using the command line I wrote, and works, this
echo "http://a.b.c/blabla/feed/" | sed -e 's#\(http://a.b.c/.*/feed/\)#\1index.xml#g'
I don't know how to transform this code so that it works in a script using -i and a file as parameter.
I tried the following but it works only if the searched string is on a line alone, while I need to transform also strings between other text. What's the correct code?
#!/bin/bash
sed -i 's#\("http://a.b.c/.*/feed/"\)#"\1index.xml"#g' $1
I think
#!/bin/bash
sed -i 's#\(http://a.b.c/.*/feed/\)#\1index.xml#g' "$1"
should work. I only removed the wrong double quotes in the command and added the missing ones around the $1.
If an input like http://aXbXc/ is not supposed to trigger the replacement, then you should also escape the dots.
I have some code using GNU-parallel which should replace text in an input file with a series of strings of the form vclist_2d_*.txt where * is a number between 1 and 10000.
FILES=(vclist_2d_*.txt)
parallel -j1 '
sed -i "s/50pc\/vclist_2d_.*/50pc\/{}'\''/" 1759_input.py
sed -i "s/schedule_analysis\/vclist_2d_.*/schedule_analysis\/{}'\\_temp\/1759_cs_output.spc''\''/" 1759_input.py
' ::: ${FILES[#]}
The first sed command successful replaces whichever vclist_2d_* file is already in 1759_input with the next one in the list FILES as defined by {}. However, the second sed command needs to replace the vclist_2d_* and append to this the text _temp/1759_cs_output.spc'
However, with the code above two things happen:
the vclist name never gets replaced with the next one in the list
the text .temp/1759_cs_output.spc gets appended rather than _temp/1759_cs_output.spc
I've tried several variations of the above none of which were successful. I'm not sure why this works successfully for the first sed but not the second. I thought maybe _ needed escaping but that didn't help.
I don't quite understand what you're doing with the single quotes: I am going to assume that your regex pattern is too greedy and you need to add a quote that got consumed. I'll change .* to [^']0 -- i.e. zero or more non-quote characters.
You're doing twice as much work as required: put both substitutions into a single sed call
parallel -j1 '
sed -i "
s#\(50pc\)/vclist_2d_[^'\'']*#\1/{}#
s#\(schedule_analysis\)/vclist_2d_[^'\'']*#\1/{}_temp/1759_cs_output.spc#
" 1759_input.py
' ::: "${FILES[#]}"
I used a different delimiter for the s/// command in order to reduce backslashes
I am trying to print out a specific section of a file which I have determined using line numbers, but the line numbers will vary from day to day so I need to be able to get the line numbers, store them as variables, then use sed to cut the lines from the stored file.
Here's what I have so far:
start-loader is a file that contains the lines I want to print, but also contains a lot of junk.
I can use sed -n '93,109p' start-loader to print out what I need, but what I want to do is this:
sed -n '$FL,$LFp' start-loader
where the variables are the line numbers I've stored.
I know that the above is not proper syntax, from a lot of research on the matter, but everything I've used either returns an error or does not work. I've tried double quotes, single then double for variables, braces for variables, and a few other things along with numerous different syntax styles. Would anyone happen to know how I can properly do this?
You need to separate the p command from your last-line variable somehow. Either of the following should work:
$ sed -n "$FL,${LF}p" start-loader
$ sed -n "$FL,$LF p" start-loader
Without the separation, the shell would try to expand the variable LFp, which does not exist, resulting in an empty string being passed to sed and causing a syntax error.
You also need to use double-quotes, not single-quotes, to allow the variables to be expanded before sed sees them.
I hope the example below answered your question:
kent$ s=3
kent$ e=8
kent$ seq 20 |sed -n "$s,$e p"
3
4
5
6
7
8