This question already has answers here:
How do I split a string on a delimiter in Bash?
(37 answers)
Closed 6 years ago.
I have a .txt file that has a list containing a hash and a password so it looks likes this:
00608cbd5037e18593f96995a080a15c:9e:hoboken
00b108d5d2b5baceeb9853b1ad9aa9e5:1c:wVPZ
Out of this txt file I need to extract only the passwords and add them in a new text file so that I have a list that would look like this:
hoboken
wVPZ
etc
etc
etc
etc
How to do this in bash, a scripting language or simply with a text editor?
Given your examples, the following use of cut would achieve what you want:
cut -f3 -d':' /folder/file >> /folder/result
The code above would delete anything before (and including) the second colon : on each line, which would work on your case, given your examples. The result is stored on /folder/result.
Edit: I edited this answer to make it simpler.
I suggest to use awk to get always last column from your file:
awk -F ':' '{print $NF}' file
Output:
hoboken
wVPZ
With sed, to remove the string up to ::
sed 's/.*://' file
You could also use grep:
$ grep -o [^:]*$ file
hoboken
wVPZ
-o print only matching part
[^:] anything but :
* all matching characters
$ end of record
Related
This question already has answers here:
How to invert a grep expression
(5 answers)
Closed 3 years ago.
So I've got a CSV file, test.csv, with the following:
Apples
Pears
Oranges
They would be comma separated given its a csv file.
I have a variables
test="app"
I want to remove anything from the csv file with that variable contents in it. So the output would be in a new file test1.csv:
Pears
Oranges
You can use sed to do that.
If you want to remove the entire row that contains test variable's value. Then try-
user#localhost$ sed "/$test/d" test.csv > output.csv
If you want to replace the word that holds the var test, then try -
user#localhost$ replacewith=''
user#localhost$ sed "s/$test/$replacewith/g" test.csv > output.csv
NB: Replace and Delete can mean same thing if you use blank string as replacement.
You can obtain such a result with sed:
sed '/dog/d' inputfile.csv > outputfile.csv
You can also make it case insensitive:
sed '/[dD][oO][gG]/d' inputfile.csv > outputfile.csv
The syntax used in the latter is RegEx, read more about it here.
This question already has answers here:
How to find patterns across multiple lines using grep?
(28 answers)
Closed 6 years ago.
I have a file that contains a sequence already broken into lines, something like this:
CGCCCATGGGTCGTATACGTAATGGGAAAACAAAGCATGGTGTAACTATGGTAAGTGCTA
GACAATACAAGAAGGCTGATATTTGTAGAATAATTCATTTGAATTATTATGCTGTAAATA
GCTAGATTATTATGCATAATTACTTTGAGAGGTGATCAATCAATTCGACCCTTGCCAATT
I want to search a specific pattern in this file like GCTGTAAATAGCTAGATTA for example.
The problem is that the pattern may be cut by a newline at an unpredictable place.
I can use :
grep -e "pattern" file
but it cannot avoid "new line" character and doesn't give the result. How can I modify my command to ignore \n in my search?
Edit:
I don't know either my query exists in the file or not, and if it is there, I don't know where it exists.
The best solution that came into my mind is
tr -d '\n' < file | grep -e "CTACCCCAGACAAACTGGTCAGATACCAACCATCAGCGAAACTAACCAAACAAA"
but I know there should be more efficient ways to do that.
pattern="GCTGTAAATA"$'\n'"GCTAGATTA" # $'\n' is Bash's way of mentioning special chars
grep -e "$pattern" file
OR
pattern="GCTGTAAATA
GCTAGATTA" # with an actual newline at the end of the first line
grep -e "$pattern" file
For satisfying a legacy code i had to add date to a filename like shown below(its definitely needed and cannot modify legacy code :( ). But i need to remove the date within the same command without going to a new line. this command is read from a text file so i should do this within the single command.
$((echo "$file_name".`date +%Y%m%d`| sed 's/^prefix_//')
so here i am removing the prefix from filename and adding a date appended to filename. i also do want to remove the date which i added. for ex: prefix_filename.txt or prefix_filename.zip should give me as below.
Expected output:
filename.txt
filename.zip
Current output:
filename.txt.20161002
filename.zip.20161002
Assumming all the files are formatted as filename.ext.date, You can pipe the output to 'cut' command and get only the 1st and 2nd fields :
~> X=filename.txt.20161002
~> echo $X | cut -d"." -f1,2
filename.txt
I am not sure that I understand your question correctly, but perhaps this does what you want:
$((echo "$file_name".`date +%Y%m%d`| sed -e 's/^prefix_//' -e 's/\.[^.]*$//')
Sample input:
cat sample
prefix_original.txt.log.tgz.10032016
prefix_original.txt.log.10032016
prefix_original.txt.10032016
prefix_one.txt.10032016
prefix.txt.10032016
prefix.10032016
grep from start of the string till a literal dot "." followed by digit.
grep -oP '^.*(?=\.\d)' sample
prefix_original.txt.log.tgz
prefix_original.txt.log
prefix_original.txt
prefix_one.txt
prefix.txt
prefix
perhaps, following should be used:
grep -oP '^.*(?=\.\d)|^.*$' sample
If I understand your question correctly, you want to remove the date part from a variable, AND you already know from the context that the variable DOES contain a date part and that this part comes after the last period in the name.
In this case, the question boils down to removing the last period and what comes after.
This can be done (Posix shell, bash, zsh, ksh) by
filename_without=${filename_with%.*}
assuming that filename_with contains the filename which has the date part in the end.
% cat example
filename.txt.20161002
filename.zip.20161002
% cat example | sed "s/.[0-9]*$//g"
filename.txt
filename.zip
%
This question already has answers here:
Bash, Linux, Need to remove lines from one file based on matching content from another file
(3 answers)
Closed 7 years ago.
I have a list of values in one file:
item2
item3
item4
and I want to remove the entire line from another file when the rows looks like this:
item1|XXXX|ABCD
item2|XXXX|ABCD
item3|XXXX|ABCD
item4|XXXX|ABCD
item5|XXXX|ABCD
So that I'm left with:
item1|XXXX|ABCD
item5|XXXX|ABCD
Is there a bash sequence to do this?
grep -vf can do the job:
grep -vFf file1 file2
item1|XXXX|ABCD
item5|XXXX|ABCD
awk to the rescue!
$ awk -F"|" 'NR==FNR{a[$1];next} !($1 in a)' remove items
item1|XXXX|ABCD
item5|XXXX|ABCD
where the item list to be removed is in file "remove" and data in file "items"
If your distinctive marker is that |XXXX|ABCD| string, you can just grep it out:
$ grep -vF '|XXXX|ABCD|' input > output
It's safer to use option -F (fixed strings) because your pattern is dangerously close to containing regex metacharacters (namely in your case: |—it's not active in the default grep regex syntax, but you don't want to worry about that when you're working with simple patterns).
If your distinctive pattern is the rest of the line, you can use a whole file as a pattern list with grep's -f option:
$ grep -vFf item_list < input > output
This question already has answers here:
Parsing variables from config file in Bash
(7 answers)
Closed 8 years ago.
I'm new to bash scripting, my task is to read a line one of my configuration file,
this is the way....I want...
let's assume "sample.conf" is the file, in file there is a line..
like this,
webURL: "http://www.sampledomain:8080"
What I want is , I need to get the value of webURL, that means "http://www.sampledomain:8080" then if the URL is not equal to "http://www.sampledomain:8080" this I need to changed it with the correct value. Hope u will help me, any help would be greatly appreciated.
Thank You.
Use awk like this:
URL=$(awk -F\" '/^webURL/{print $2}' sample.conf)
echo $URL
http://www.sampledomain:8080
The $() means "put the result of running the enclosed command into the variable URL". The awk then sets the separator for fields to the double quote sign. It then looks in your file for a line that starts with "webURL" and when it finds it, it outputs everything between the second pair of double quotes, i.e. field 2 on the line.
EDITED to answer further question
If you want to parse out a subscribe_key value from a java file called thing.java, that looks like this:
subscribe_key = 'sub-f-xx-xx-xx-xx-xx';
you can use this:
key=$(awk -F\' '/^subscribe_key/{print $2}' thing.java)
echo $key
sub-f-xx-xx-xx-xx-xx
Note that I changed the double quote after -F to single quote to tell awk that fields are now separated by single quotes.
Note that if you have values that are marked by single quotes and double quotes IN THE SAME FILE you can tell awk to use either single quotes or double quotes as field separators like this:
value=$(awk -F"\'|\"" '/^subscribe_key/{print $2}' yourFile)
echo $value
sub-f-xx-xx-xx-xx-xx
value=$(awk -F"\'|\"" '/^webURL/{print $2}' yourFile)
echo $value
http://www.sampledomain:8080
url=`grep webURL sample.conf | sed -e 's/webURL: //'`
using sed
sed 's#\(webURL: \).*#\1 "http://www.sampledomain:8080"#' file