I have a file full of IDs which I need to use to build a list of URLs as part of a bash file.
ids.txt is as follows:
s_Foo
p_Bar
s1_Blah
e_Yah
The URLs will always end in a filename that contains the ID, in its own path.
I've looked around for how to prepend and append using sed, but cannot figure out to do the duplicating copy/paste part (\1) using that tool. The ID can be anything, so pattern matching seems hard. Duplication of everything before the line break seems more sensible? I don't know.
How do I create something like this as urls.txt using sed or awk? Is it possible?
https://link.domain.com/list/s_Foo/s_Foo_meta.xml
https://link.domain.com/list/p_Bar/p_Bar_meta.xml
https://link.domain.com/list/s1_Blah/s1_Blah_meta.xml
https://link.domain.com/list/e_Yah/e_Yah_meta.xml
$ sed 's#.*#https://link.domain.com/list/&/&_meta.xml#' ids.txt
https://link.domain.com/list/s_Foo/s_Foo_meta.xml
https://link.domain.com/list/p_Bar/p_Bar_meta.xml
https://link.domain.com/list/s1_Blah/s1_Blah_meta.xml
https://link.domain.com/list/e_Yah/e_Yah_meta.xml
$ awk '{sub(/.*/,"https://link.domain.com/list/&/&_meta.xml")}1' ids.txt
https://link.domain.com/list/s_Foo/s_Foo_meta.xml
https://link.domain.com/list/p_Bar/p_Bar_meta.xml
https://link.domain.com/list/s1_Blah/s1_Blah_meta.xml
https://link.domain.com/list/e_Yah/e_Yah_meta.xml
try gnu sed:
sed -E 's/\S+/https://link.domain.com/list/&/&_meta.xml' ids.txt >urls.txt
{"success":true,"message":"","result":[{"Id":66719299,"TimeStamp":"2017-08-29T16:35:35.937","Quantity":0.80122000,"Price":0.01379000,"Total":0.01104882,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719283,"TimeStamp":"2017-08-29T16:35:32.36","Quantity":7.36427025,"Price":0.01379000,"Total":0.10155328,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719222,"TimeStamp":"2017-08-29T16:35:13.263","Quantity":24.03098850,"Price":0.01379000,"Total":0.33138733,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719221,"TimeStamp":"2017-08-29T16:35:13.013","Quantity":0.70000000,"Price":0.01379000,"Total":0.00965300,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719220,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":31.30579055,"Price":0.01379000,"Total":0.43170685,"FillType":"PARTIAL_FILL","OrderType":"BUY"},{"Id":66719219,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":12.87144703,"Price":0.01378000,"Total":0.17736854,"FillType":"PARTIAL_FILL","OrderType":"BUY"}]}
Im stuck on how I would use sed to remove anything after 66719222 including and then adding ]} at the end. The output should be
This is what I want to delete:
,{"Id":66719222,"TimeStamp":"2017-08-29T16:35:13.263","Quantity":24.03098850,"Price":0.01379000,"Total":0.33138733,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719221,"TimeStamp":"2017-08-29T16:35:13.013","Quantity":0.70000000,"Price":0.01379000,"Total":0.00965300,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719220,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":31.30579055,"Price":0.01379000,"Total":0.43170685,"FillType":"PARTIAL_FILL","OrderType":"BUY"},{"Id":66719219,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":12.87144703,"Price":0.01378000,"Total":0.17736854,"FillType":"PARTIAL_FILL","OrderType":"BUY"}]}
and replace it with ]}
so the output is
{"success":true,"message":"","result":[{"Id":66719299,"TimeStamp":"2017-08-29T16:35:35.937","Quantity":0.80122000,"Price":0.01379000,"Total":0.01104882,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719283,"TimeStamp":"2017-08-29T16:35:32.36","Quantity":7.36427025,"Price":0.01379000,"Total":0.10155328,"FillType":"PARTIAL_FILL","OrderType":"SELL"}]}
Thanks if anyone could help.
sed 's/,{"Id":66719222.*/]}/' filename
Or if you're willing to sacrifice clarity to make the command one character shorter:
sed 's/,[^}]*66719222.*/]}/' filename
Or more crudely:
sed 's/.\{7\}66719222.*/]}/' filename
Assuming file is your filename.
sed 's/\(.*\),{"Id":66719222.*/\1\]}/' file
If it is in bash, why not just use bash:
str='{"success":true,"message":"","result":[{"Id":66719299,"TimeStamp":"2017-08-29T16:35:35.937","Quantity":0.80122000,"Price":0.01379000,"Total":0.01104882,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719283,"TimeStamp":"2017-08-29T16:35:32.36","Quantity":7.36427025,"Price":0.01379000,"Total":0.10155328,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719222,"TimeStamp":"2017-08-29T16:35:13.263","Quantity":24.03098850,"Price":0.01379000,"Total":0.33138733,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719221,"TimeStamp":"2017-08-29T16:35:13.013","Quantity":0.70000000,"Price":0.01379000,"Total":0.00965300,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719220,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":31.30579055,"Price":0.01379000,"Total":0.43170685,"FillType":"PARTIAL_FILL","OrderType":"BUY"},{"Id":66719219,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":12.87144703,"Price":0.01378000,"Total":0.17736854,"FillType":"PARTIAL_FILL","OrderType":"BUY"}]}'
echo "${x%66719222*}66719222]}"
My csv file is:
1234,0045,0045,0345,300
1235,0046,0446,2345,301
1236,0047,0447,0350,302
I need the result like this:
1234,45,45,345,300
1235,46,446,2345,301
1236,47,447,350,302
I want to accomplish this task using Unix SED command. Any other alternative method is also appreciated.
Try this :
$ sed 's/^0*\([^,]\)/\1/;s/,0*\([^,]\)/,\1/g' yourfile.csv
1234,45,45,345,300
1235,46,446,2345,301
1236,47,447,350,302
Input -
ajflkajlkaj
ddddddddddd/need/to/print/thiskkkkkkkkkkkk
ahfkjahkfja
Output needed
/need/to/print/this
Googling and tweaking I was able to extract "this" like by doing this-
sed -n 's/^.*\(this\).*$/\1/p' myfile
but can't seem to get the syntax right for getting the rest of the pattern starting from from "/need".
I would appreciate any pointers/suggestions.
Thanks in advance.
You need to let us know what the ddd and kkk really stand for. Assuming there is some white space behind "this", you can match this pattern instead:
sed -n 's/^[^\/]*\(.*\)\s.*$/\1/p' myfile
[^\/]* means match as many non-/ characters first.
Try this :
sed -n 's![^/]\+\(.*/this\).*!\1!p' filename.txt
NOTE
the sed separator can be what you want instead of the default /. I use ! here.
If you prefer, a simple grep command will works too :
grep --binary-files=text -o '/.*/this' filename.txt
I'm looking for a way to remove lines within multiple csv files, in bash using sed, awk or anything appropriate where the file ends in 0.
So there are multiple csv files, their format is:
EXAMPLEfoo,60,6
EXAMPLEbar,30,10
EXAMPLElong,60,0
EXAMPLEcon,120,6
EXAMPLEdev,60,0
EXAMPLErandom,30,6
So the file will be amended to:
EXAMPLEfoo,60,6
EXAMPLEbar,30,10
EXAMPLEcon,120,6
EXAMPLErandom,30,6
A problem which I can see arising is distinguishing between double digits that end in zero and 0 itself.
So any ideas?
Using your file, something like this?
$ sed '/,0$/d' test.txt
EXAMPLEfoo,60,6
EXAMPLEbar,30,10
EXAMPLEcon,120,6
EXAMPLErandom,30,6
For this particular problem, sed is perfect, as the others have pointed out. However, awk is more flexible, i.e. you can filter on an arbitrary column:
awk -F, '$3!=0' test.csv
This will print the entire line is column 3 is not 0.
use sed to only remove lines ending with ",0":
sed '/,0$/d'
you can also use awk,
$ awk -F"," '$NF!=0' file
EXAMPLEfoo,60,6
EXAMPLEbar,30,10
EXAMPLEcon,120,6
EXAMPLErandom,30,6
this just says check the last field for 0 and don't print if its found.
sed '/,[ \t]*0$/d' file
I would tend to sed, but there is an egrep (or: grep -e) -solution too:
egrep -v ",0$" example.csv