Remove leading Zero in CSV file using Unix sed command - bash

My csv file is:
1234,0045,0045,0345,300
1235,0046,0446,2345,301
1236,0047,0447,0350,302
I need the result like this:
1234,45,45,345,300
1235,46,446,2345,301
1236,47,447,350,302
I want to accomplish this task using Unix SED command. Any other alternative method is also appreciated.


Try this :
$ sed 's/^0*\([^,]\)/\1/;s/,0*\([^,]\)/,\1/g' yourfile.csv
1234,45,45,345,300
1235,46,446,2345,301
1236,47,447,350,302

Related

Search and copy a specific part of a file to another file in Shell Scripting

Suppose I have a text file with a lot of data. Using Shell Scripting, while reading the file if I get the keyword "CURRENT" it will copy the data to another file until it is getting the keyword "END-0".
Not able to develop the script snippet for this.

You probably want to look at awk or sed for this task.
# Using SED
sed -n '/CURRENT/,/END-0/p' input-file >another-file
# Using AWK
awk '/CURRENT/,/END-0/' input-file >another-file

Prepend, copy/paste, and append using sed?

I have a file full of IDs which I need to use to build a list of URLs as part of a bash file.
ids.txt is as follows:
s_Foo
p_Bar
s1_Blah
e_Yah
The URLs will always end in a filename that contains the ID, in its own path.
I've looked around for how to prepend and append using sed, but cannot figure out to do the duplicating copy/paste part (\1) using that tool. The ID can be anything, so pattern matching seems hard. Duplication of everything before the line break seems more sensible? I don't know.
How do I create something like this as urls.txt using sed or awk? Is it possible?
https://link.domain.com/list/s_Foo/s_Foo_meta.xml
https://link.domain.com/list/p_Bar/p_Bar_meta.xml
https://link.domain.com/list/s1_Blah/s1_Blah_meta.xml
https://link.domain.com/list/e_Yah/e_Yah_meta.xml

$ sed 's#.*#https://link.domain.com/list/&/&_meta.xml#' ids.txt
https://link.domain.com/list/s_Foo/s_Foo_meta.xml
https://link.domain.com/list/p_Bar/p_Bar_meta.xml
https://link.domain.com/list/s1_Blah/s1_Blah_meta.xml
https://link.domain.com/list/e_Yah/e_Yah_meta.xml
$ awk '{sub(/.*/,"https://link.domain.com/list/&/&_meta.xml")}1' ids.txt
https://link.domain.com/list/s_Foo/s_Foo_meta.xml
https://link.domain.com/list/p_Bar/p_Bar_meta.xml
https://link.domain.com/list/s1_Blah/s1_Blah_meta.xml
https://link.domain.com/list/e_Yah/e_Yah_meta.xml

try gnu sed:
sed -E 's/\S+/https://link.domain.com/list/&/&_meta.xml' ids.txt >urls.txt

Bash Script how to use sed to replace text after a number?

{"success":true,"message":"","result":[{"Id":66719299,"TimeStamp":"2017-08-29T16:35:35.937","Quantity":0.80122000,"Price":0.01379000,"Total":0.01104882,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719283,"TimeStamp":"2017-08-29T16:35:32.36","Quantity":7.36427025,"Price":0.01379000,"Total":0.10155328,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719222,"TimeStamp":"2017-08-29T16:35:13.263","Quantity":24.03098850,"Price":0.01379000,"Total":0.33138733,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719221,"TimeStamp":"2017-08-29T16:35:13.013","Quantity":0.70000000,"Price":0.01379000,"Total":0.00965300,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719220,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":31.30579055,"Price":0.01379000,"Total":0.43170685,"FillType":"PARTIAL_FILL","OrderType":"BUY"},{"Id":66719219,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":12.87144703,"Price":0.01378000,"Total":0.17736854,"FillType":"PARTIAL_FILL","OrderType":"BUY"}]}
Im stuck on how I would use sed to remove anything after 66719222 including and then adding ]} at the end. The output should be
This is what I want to delete:
,{"Id":66719222,"TimeStamp":"2017-08-29T16:35:13.263","Quantity":24.03098850,"Price":0.01379000,"Total":0.33138733,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719221,"TimeStamp":"2017-08-29T16:35:13.013","Quantity":0.70000000,"Price":0.01379000,"Total":0.00965300,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719220,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":31.30579055,"Price":0.01379000,"Total":0.43170685,"FillType":"PARTIAL_FILL","OrderType":"BUY"},{"Id":66719219,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":12.87144703,"Price":0.01378000,"Total":0.17736854,"FillType":"PARTIAL_FILL","OrderType":"BUY"}]}
and replace it with ]}
so the output is
{"success":true,"message":"","result":[{"Id":66719299,"TimeStamp":"2017-08-29T16:35:35.937","Quantity":0.80122000,"Price":0.01379000,"Total":0.01104882,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719283,"TimeStamp":"2017-08-29T16:35:32.36","Quantity":7.36427025,"Price":0.01379000,"Total":0.10155328,"FillType":"PARTIAL_FILL","OrderType":"SELL"}]}
Thanks if anyone could help.

sed 's/,{"Id":66719222.*/]}/' filename
Or if you're willing to sacrifice clarity to make the command one character shorter:
sed 's/,[^}]*66719222.*/]}/' filename
Or more crudely:
sed 's/.\{7\}66719222.*/]}/' filename

Assuming file is your filename.
sed 's/\(.*\),{"Id":66719222.*/\1\]}/' file

If it is in bash, why not just use bash:
str='{"success":true,"message":"","result":[{"Id":66719299,"TimeStamp":"2017-08-29T16:35:35.937","Quantity":0.80122000,"Price":0.01379000,"Total":0.01104882,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719283,"TimeStamp":"2017-08-29T16:35:32.36","Quantity":7.36427025,"Price":0.01379000,"Total":0.10155328,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719222,"TimeStamp":"2017-08-29T16:35:13.263","Quantity":24.03098850,"Price":0.01379000,"Total":0.33138733,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719221,"TimeStamp":"2017-08-29T16:35:13.013","Quantity":0.70000000,"Price":0.01379000,"Total":0.00965300,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719220,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":31.30579055,"Price":0.01379000,"Total":0.43170685,"FillType":"PARTIAL_FILL","OrderType":"BUY"},{"Id":66719219,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":12.87144703,"Price":0.01378000,"Total":0.17736854,"FillType":"PARTIAL_FILL","OrderType":"BUY"}]}'
echo "${x%66719222*}66719222]}"

Unix Find & Replace Issue

I have a pretty huge .csv file with the date at column 3 ( Example: 11/17/2015) and i need to replace with the date format as 2015-11-17. I tried doing using:%s/\<11/17/2015\>/2015-11-17.But couldn't see the change. Any suggestions on how to do this?.

I assume you are using vim:
:%s/11\/17\/2015/2015-11-17/g
You can do this also with sed without opening the file:
sed -i 's/11\/17\/2015/2015-11-17/' somefile.csv

Try to escape backslash like:
echo "11/17/2015" | sed 's/11\/17\/2015/2015-11-17/g'
2015-11-17

replacing a pattern using sed command

I want to replace a pattern by using sed command. I have a file with below data.
50%%R39%35%R43%-35%R4Z%10%RRN%1110%R0M%-950
Now I need to replace the pattern %RRN%<something>% with %RRN%0%. output should be like this.
50%%R39%35%R43%-35%R4Z%10%RRN%0%R0M%-950
For this I have used below command
sed 's/%RRN%\(.*\)%/%RRN%0%/g'
But I didn't get the correct output. It is coming as below.
50%R39%35%R43%-35%R4Z%10%RRN%0%-950
Please help me on this

Better solution would be use [^%]\+ in place of .*
sed 's/%RRN%\([^%]\+\)%/%RRN%0%/g'
Output:
50%%R39%35%R43%-35%R4Z%10%RRN%0%R0M%-950

If you like to try with awk, you can do:
echo '50%%R39%35%R43%-35%R4Z%10%RRN%1110%R0M%-950' | awk '{sub(/%RRN%[^%]*%/,"%RRN%0%")}1'
50%%R39%35%R43%-35%R4Z%10%RRN%0%R0M%-950

perl -pe '$a="0%";s/(%RRN%)[^%]*%/$1$a/g'

Resources