Using sed to find text BEFORE a string - bash

So I'm trying to extract URLs from url files
and i need some kind of descriptor for the file--
If I have something like
param5x57389dwordhereparam3904
and I want to extract wordhere, I know I can use
$(sed -n 's/param5x57389//p' $f)
to get wordhereparam3904
so how do I go about taking off that last part?
Thanks in advance!


You can use the following sed command:
sed '/param5x57389d/ s/param5x57389d\(.*\)param.*/\1/' "$f"

Related

Prepend, copy/paste, and append using sed?

I have a file full of IDs which I need to use to build a list of URLs as part of a bash file.
ids.txt is as follows:
s_Foo
p_Bar
s1_Blah
e_Yah
The URLs will always end in a filename that contains the ID, in its own path.
I've looked around for how to prepend and append using sed, but cannot figure out to do the duplicating copy/paste part (\1) using that tool. The ID can be anything, so pattern matching seems hard. Duplication of everything before the line break seems more sensible? I don't know.
How do I create something like this as urls.txt using sed or awk? Is it possible?
https://link.domain.com/list/s_Foo/s_Foo_meta.xml
https://link.domain.com/list/p_Bar/p_Bar_meta.xml
https://link.domain.com/list/s1_Blah/s1_Blah_meta.xml
https://link.domain.com/list/e_Yah/e_Yah_meta.xml

$ sed 's#.*#https://link.domain.com/list/&/&_meta.xml#' ids.txt
https://link.domain.com/list/s_Foo/s_Foo_meta.xml
https://link.domain.com/list/p_Bar/p_Bar_meta.xml
https://link.domain.com/list/s1_Blah/s1_Blah_meta.xml
https://link.domain.com/list/e_Yah/e_Yah_meta.xml
$ awk '{sub(/.*/,"https://link.domain.com/list/&/&_meta.xml")}1' ids.txt
https://link.domain.com/list/s_Foo/s_Foo_meta.xml
https://link.domain.com/list/p_Bar/p_Bar_meta.xml
https://link.domain.com/list/s1_Blah/s1_Blah_meta.xml
https://link.domain.com/list/e_Yah/e_Yah_meta.xml

try gnu sed:
sed -E 's/\S+/https://link.domain.com/list/&/&_meta.xml' ids.txt >urls.txt

Bash read file and get values from to

Any idea how to read a file and get text from to values?
Let´s say I have
Json info lastActiveTimes:{"707514313":1505584723,"100004389551456":1505591385},chatNotif:0}, a lot of more Json info.
And I want to get from Start: to End, so read the file and just return
{"707514313":1505584723,"100004389551456":1505591385}
I´m using bash from OSX

When you have 2nd line with start in it between Start and End, you can use
sed -n '/Start:/,/End/p' file| sed '1s/.*Start://; $s/End.*//'
UPDATE:
Question has changed,now it can be anything like
sed 's/.*\({[^}]*}\).*/\1/' file
of
grep -Eo "{[^}]*}" file

With GNU grep and Perl regular expression (-P):
grep -Poz '(?<=Start:).*(\n)*.*(?= End)' file
Output:
aaaaaaaaaabbbbbbbb
ccccccc

Bash Script how to use sed to replace text after a number?

{"success":true,"message":"","result":[{"Id":66719299,"TimeStamp":"2017-08-29T16:35:35.937","Quantity":0.80122000,"Price":0.01379000,"Total":0.01104882,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719283,"TimeStamp":"2017-08-29T16:35:32.36","Quantity":7.36427025,"Price":0.01379000,"Total":0.10155328,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719222,"TimeStamp":"2017-08-29T16:35:13.263","Quantity":24.03098850,"Price":0.01379000,"Total":0.33138733,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719221,"TimeStamp":"2017-08-29T16:35:13.013","Quantity":0.70000000,"Price":0.01379000,"Total":0.00965300,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719220,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":31.30579055,"Price":0.01379000,"Total":0.43170685,"FillType":"PARTIAL_FILL","OrderType":"BUY"},{"Id":66719219,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":12.87144703,"Price":0.01378000,"Total":0.17736854,"FillType":"PARTIAL_FILL","OrderType":"BUY"}]}
Im stuck on how I would use sed to remove anything after 66719222 including and then adding ]} at the end. The output should be
This is what I want to delete:
,{"Id":66719222,"TimeStamp":"2017-08-29T16:35:13.263","Quantity":24.03098850,"Price":0.01379000,"Total":0.33138733,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719221,"TimeStamp":"2017-08-29T16:35:13.013","Quantity":0.70000000,"Price":0.01379000,"Total":0.00965300,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719220,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":31.30579055,"Price":0.01379000,"Total":0.43170685,"FillType":"PARTIAL_FILL","OrderType":"BUY"},{"Id":66719219,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":12.87144703,"Price":0.01378000,"Total":0.17736854,"FillType":"PARTIAL_FILL","OrderType":"BUY"}]}
and replace it with ]}
so the output is
{"success":true,"message":"","result":[{"Id":66719299,"TimeStamp":"2017-08-29T16:35:35.937","Quantity":0.80122000,"Price":0.01379000,"Total":0.01104882,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719283,"TimeStamp":"2017-08-29T16:35:32.36","Quantity":7.36427025,"Price":0.01379000,"Total":0.10155328,"FillType":"PARTIAL_FILL","OrderType":"SELL"}]}
Thanks if anyone could help.

sed 's/,{"Id":66719222.*/]}/' filename
Or if you're willing to sacrifice clarity to make the command one character shorter:
sed 's/,[^}]*66719222.*/]}/' filename
Or more crudely:
sed 's/.\{7\}66719222.*/]}/' filename

Assuming file is your filename.
sed 's/\(.*\),{"Id":66719222.*/\1\]}/' file

If it is in bash, why not just use bash:
str='{"success":true,"message":"","result":[{"Id":66719299,"TimeStamp":"2017-08-29T16:35:35.937","Quantity":0.80122000,"Price":0.01379000,"Total":0.01104882,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719283,"TimeStamp":"2017-08-29T16:35:32.36","Quantity":7.36427025,"Price":0.01379000,"Total":0.10155328,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719222,"TimeStamp":"2017-08-29T16:35:13.263","Quantity":24.03098850,"Price":0.01379000,"Total":0.33138733,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719221,"TimeStamp":"2017-08-29T16:35:13.013","Quantity":0.70000000,"Price":0.01379000,"Total":0.00965300,"FillType":"PARTIAL_FILL","OrderType":"SELL"},{"Id":66719220,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":31.30579055,"Price":0.01379000,"Total":0.43170685,"FillType":"PARTIAL_FILL","OrderType":"BUY"},{"Id":66719219,"TimeStamp":"2017-08-29T16:35:12.623","Quantity":12.87144703,"Price":0.01378000,"Total":0.17736854,"FillType":"PARTIAL_FILL","OrderType":"BUY"}]}'
echo "${x%66719222*}66719222]}"

How to extract a string at end of line after a specific word

I have different location, but they all have a pattern:
some_text/some_text/some_text/log/some_text.text
All locations don't start with the same thing, and they don't have the same number of subdirectories, but I am interested in what comes after log/ only. I would like to extract the .text
edited question:
I have a lot of location:
/s/h/r/t/log/b.p
/t/j/u/f/e/log/k.h
/f/j/a/w/g/h/log/m.l
Just to show you that I don't know what they are, the user enters these location, so I have no idea what the user enters. The only I know is that it always contains log/ followed by the name of the file.
I would like to extract the type of the file, whatever string comes after the dot

THe only i know is that it always contains log/ followed by the name
of the file.
I would like to extract the type of the file, whatever string comes
after the dot
based on this requirement, this line works:
grep -o '[^.]*$' file
for your example, it outputs:
text

You can use bash built-in string operations. The example below will extract everything after the last dot from the input string.
$ var="some_text/some_text/some_text/log/some_text.text"
$ echo "${var##*.}"
text
Alternatively, use sed:
$ sed 's/.*\.//' <<< "$var"
text

Not the cleanest way, but this will work
sed -e "s/.*log\///" | sed -e "s/\..*//"
This is the sed patterns for it anyway, not sure if you have that string in a variable, or if you're reading from a file etc.
You could also grab that text and store in a sed register for later substitution etc. All depends on exactly what you are trying to do.

Using awk
awk -F'.' '{print $NF}' file
Using sed
sed 's/.*\.//' file

Running from the root of this structure:
/s/h/r/t/log/b.p
/t/j/u/f/e/log/k.h
/f/j/a/w/g/h/log/m.l
This seems to work, you can skip the echo command if you really just want the file types with no record of where they came from.
$ for DIR in *; do
> echo -n "$DIR "
> find $DIR -path "*/log/*" -exec basename {} \; | sed 's/.*\.//'
> done
f l
s p
t h

Printing only the matched pattern between <start> and <end> in sed

Input -
ajflkajlkaj
ddddddddddd/need/to/print/thiskkkkkkkkkkkk
ahfkjahkfja
Output needed
/need/to/print/this
Googling and tweaking I was able to extract "this" like by doing this-
sed -n 's/^.*\(this\).*$/\1/p' myfile
but can't seem to get the syntax right for getting the rest of the pattern starting from from "/need".
I would appreciate any pointers/suggestions.
Thanks in advance.

You need to let us know what the ddd and kkk really stand for. Assuming there is some white space behind "this", you can match this pattern instead:
sed -n 's/^[^\/]*\(.*\)\s.*$/\1/p' myfile
[^\/]* means match as many non-/ characters first.

Try this :
sed -n 's![^/]\+\(.*/this\).*!\1!p' filename.txt
NOTE
the sed separator can be what you want instead of the default /. I use ! here.
If you prefer, a simple grep command will works too :
grep --binary-files=text -o '/.*/this' filename.txt

Resources