This question already has answers here:
How to invert a grep expression
(5 answers)
Closed 3 years ago.
So I've got a CSV file, test.csv, with the following:
Apples
Pears
Oranges
They would be comma separated given its a csv file.
I have a variables
test="app"
I want to remove anything from the csv file with that variable contents in it. So the output would be in a new file test1.csv:
Pears
Oranges
You can use sed to do that.
If you want to remove the entire row that contains test variable's value. Then try-
user#localhost$ sed "/$test/d" test.csv > output.csv
If you want to replace the word that holds the var test, then try -
user#localhost$ replacewith=''
user#localhost$ sed "s/$test/$replacewith/g" test.csv > output.csv
NB: Replace and Delete can mean same thing if you use blank string as replacement.
You can obtain such a result with sed:
sed '/dog/d' inputfile.csv > outputfile.csv
You can also make it case insensitive:
sed '/[dD][oO][gG]/d' inputfile.csv > outputfile.csv
The syntax used in the latter is RegEx, read more about it here.
Related
This question already has answers here:
Bash: Parse CSV with quotes, commas and newlines
(10 answers)
Closed 2 years ago.
this is sample csv file :
name,annotations,description
drgreghouse,princeton,"doctor,head"
sheldon,tbbt,"physicist,actor"
chandler,friends,"actor,comedian"
I am trying something like this but it is reading only first values
INPUT="$(pwd)/data.csv"
IFS=','
sed 1d $INPUT |while read name annotations description; do
echo "$name $annotations $description"
done
O/p -
drgreghouse princeton "doctor
sheldon tbbt "physicist
chandler friends "actor
Expected O/p
drgreghouse princeton doctor,head
sheldon tbbt physicist,actor
chandler friends actor,comedian
sed and the shell have no concept of CSV files. If you want to process quoted fields in CSV, you have to handle the quoted fields yourself, or switch to a tool which handles them for you.
If your data has no complex quoting, you can replace every non-quoted comma with a different delimiter which doesn't occur in the data (try | maybe) and take it from there. If you just want to convert to space-delimited, try this.
sed -e 1d -e 's/"\([^"]*\)",\|\([^[",]*\),/\1\2 /g' data.csv
Demo: https://ideone.com/sg9crO
To use a different delimiter, change the space after \1\2 to that delimiter. But again, please understand that this quick and dirty regex hack cannot handle the full range of CSV's quoting rules.
As an aside, you don't need $(pwd) to refer to the current directory; relative file names are always resolved relative to your current working directory already.
This question already has answers here:
In-place edits with sed on OS X
(8 answers)
Closed 2 years ago.
For example, I want do remove all lines in a textile that do not contain the character '#'
I have already tried to use sed like so
sed '/#/!d' data.txt
What am I missing? Shouldn't this work?
I prefer using ed over the non-standard sed -i, especially if it needs to be portable:
printf "%s\n" "v/#/d" w | ed -s filename
This deletes every line that doesn't contain a #, and saves the changed file back to disc.
sed -n '/#/p' [file]
-n suppress default printing
/#/ match on # anywhere on the line
p print if it matches
Add -i for in-place editing of the file (if supplied).
This question already has answers here:
How do I split a string on a delimiter in Bash?
(37 answers)
Closed 6 years ago.
I have a .txt file that has a list containing a hash and a password so it looks likes this:
00608cbd5037e18593f96995a080a15c:9e:hoboken
00b108d5d2b5baceeb9853b1ad9aa9e5:1c:wVPZ
Out of this txt file I need to extract only the passwords and add them in a new text file so that I have a list that would look like this:
hoboken
wVPZ
etc
etc
etc
etc
How to do this in bash, a scripting language or simply with a text editor?
Given your examples, the following use of cut would achieve what you want:
cut -f3 -d':' /folder/file >> /folder/result
The code above would delete anything before (and including) the second colon : on each line, which would work on your case, given your examples. The result is stored on /folder/result.
Edit: I edited this answer to make it simpler.
I suggest to use awk to get always last column from your file:
awk -F ':' '{print $NF}' file
Output:
hoboken
wVPZ
With sed, to remove the string up to ::
sed 's/.*://' file
You could also use grep:
$ grep -o [^:]*$ file
hoboken
wVPZ
-o print only matching part
[^:] anything but :
* all matching characters
$ end of record
This question already has answers here:
How to append a string at end of a specific line in a file in bash [duplicate]
(4 answers)
Closed 6 years ago.
I am trying to achieve the following with bash:
Create a file called "file" which contains the following data:
line1
line2
line3456
I want to append additional characters after "line2", for example:
line1
line2 test123
line3456
I know that I can use sed to replace the whole line (sed -i 's/line2/line2 test123/' file). But I am sure that there is a better way to do it.
I intend to modify lines with more than one word in a line so maybe I will need wildcards as well.
sed 's/line2/& test123/' file
If you want to edit your file "in place" use sed's option -i.
Use & as back-reference to the matched pattern:
sed -i 's/word1 word2 word3/& test123/' file
This wil append text " test123" in a line that has word1 word2 word3 text in it.
This question already has answers here:
Bash, Linux, Need to remove lines from one file based on matching content from another file
(3 answers)
Closed 7 years ago.
I have a list of values in one file:
item2
item3
item4
and I want to remove the entire line from another file when the rows looks like this:
item1|XXXX|ABCD
item2|XXXX|ABCD
item3|XXXX|ABCD
item4|XXXX|ABCD
item5|XXXX|ABCD
So that I'm left with:
item1|XXXX|ABCD
item5|XXXX|ABCD
Is there a bash sequence to do this?
grep -vf can do the job:
grep -vFf file1 file2
item1|XXXX|ABCD
item5|XXXX|ABCD
awk to the rescue!
$ awk -F"|" 'NR==FNR{a[$1];next} !($1 in a)' remove items
item1|XXXX|ABCD
item5|XXXX|ABCD
where the item list to be removed is in file "remove" and data in file "items"
If your distinctive marker is that |XXXX|ABCD| string, you can just grep it out:
$ grep -vF '|XXXX|ABCD|' input > output
It's safer to use option -F (fixed strings) because your pattern is dangerously close to containing regex metacharacters (namely in your case: |—it's not active in the default grep regex syntax, but you don't want to worry about that when you're working with simple patterns).
If your distinctive pattern is the rest of the line, you can use a whole file as a pattern list with grep's -f option:
$ grep -vFf item_list < input > output