I am trying to write a bash script that uses sed to modify lines in a config file not containing a specific string. To illustrate by example, I could have ...
/some/file/path1 ipAddress1/subnetMask(rw,sync,no_root_squash)
/some/file/path2 ipAddress1/subnetMask(rw,sync,no_root_squash,anonuid=-1)
/some/file/path3 ipAddress2/subnetMask(rw,sync,no_root_squash,anonuid=0)
/some/file/path4 ipAddress2/subnetMask(rw,sync,no_root_squash,anongid=-1)
/some/file/path5 ipAddress2/subnetMask(rw,sync,no_root_squash,anonuid=-1,anongid=-1)
And I want every line's parenthetical list to be changed such that it contains strings anonuid=-1 and anongid=-1 within its parentheses ...
/some/file/path1 ipAddress1/subnetMask(rw,sync,no_root_squash,anonuid=-1,anongid=-1)
/some/file/path2 ipAddress1/subnetMask(rw,sync,no_root_squash,anonuid=-1,anongid=-1)
/some/file/path3 ipAddress2/subnetMask(rw,sync,no_root_squash,anonuid=-1,anongid=-1)
/some/file/path4 ipAddress2/subnetMask(rw,sync,no_root_squash,anongid=-1,anonuid=-1)
/some/file/path5 ipAddress2/subnetMask(rw,sync,no_root_squash,anonuid=-1,anongid=-1)
As can be seen from the example, both anonuid and anongid may already exist within the parentheses, but it is possible that the original parenthetical list has one string but not the other (lines 2, 3, and 4), the list has neither (line 1), the list has both already set properly (line 5), or even one or both of them are set incorrectly (line 3). When either anonuid or anongid is set to a value other than -1, it must be changed to the proper value of -1 (line 3).
What would be the best way to edit my config file using sed such that anonuid=-1 and anongid=-1 is contained in each line's parenthetical list, separated by a comma delimiter of course?
I think this does what you want:
sed -e '/anonuid/{s/anonuid=[-0-9]*/anonuid=-1/;b gid;};s/)$/,anonuid=-1)/;:gid;/anongid/{s/anongid=[-0-9]*/anongid=-1/;b;};s/)$/,anongid=-1)/'
Basically, it has two nearly identical parts with the first dealing with anonuid and the second anongid, each with a bit of logic to decide if it needs to replace or add the appropriate values. (It doesn't bother to check if the value is already correct, that would just complicate things while not changing the results.)
You can use sed to specify the lines you are interested in:
$ sed '/anonuid=..*,anongid=..*)$/!p' $file
The above will print (p) all lines that don't match the regular expression between the two slashes. I negated the expression by using the !. This way, you're not matching lines with both anaonuid and anongid in them.
Now, you can work on the non-matching lines and editing those with the sed s command:
$ sed '/anonuid=..*,anongid=..*)$/!s/from/to/`
The manipulation might be fairly complex, and you might be passing multiple sed commands to get everything just right.
However, if the string no_root_squash appear in each line you want to change, why not take the simple way out:
$ sed 's/no_root_squash.*$/no_root_squash,anonuid=-1,anongid=-1)/' $file
This is looking for that no_root_squash string, and replacing everything from that string to the end of the line with the text you want. Are there lines you are touching that don't need to be edited? Yes, but you're not really changing those lines. You're basically substituting /no_root_squash,anonuid=-1,anongid=-1) with the same /no_root_squash,anonuid=-1,anongid=-1).
This may be faster even though it's replacing text that doesn't need replacing because there's less processing going on. Plus, it's easier to understand and support in the future.
Response
Thanks David! Yeah I was considering going that route, but I didn't want to rely 100% on every line containing no_root_squash. My current config file only ends in that string, but I'm just not 100% sure that won't potentially be different in the field. Do you think there would be a way to change that so it just overwrites from the end of the last string not containing anonuid=-1 or anongid=-1 onward?
What can you guarantee will be in each line?
You might be able to do a capture group:
sed 's/\(sync,[^,)]*\).*/\1,anonuid=-1,anongid=-1)/' $file
The \(..\) is a capture group. It basically captures that portion of the matching regular expression, and then allows you to reuse it via the \1. I'm capturing from the word sync to a group of characters not including a comma or a closing parentheses. Then, I'm appending the capture group, a comma, and your anon uid and gid.
Will that work?
Maybe I am oversimplifying:
sed 's/anonuid=[-0-9]*[^)]//g;s/anongid=[-0-9]*[^)]//g;s/[)]/anonuid=-1,anongid=-1)/g' test.txt > test3.txt
This just drops any current instance of anonuid or anongid and adds the string
"anonuid=-1,anongid=-1" into the parentheses
Related
This question already has answers here:
change date format from DD/MM/YYYY to YYYY-MM-DD with sed
(3 answers)
Closed last year.
This post was edited and submitted for review last year and failed to reopen the post:
Original close reason(s) were not resolved
I have files with dates and times formatted as such and about 6 million lines of them
20211231233710;20211231233713;SomeHEXID;SomeIDNumber;Whatever;0;SomethingElse
How can I make it so that the dates are formatted in this way?
2021-12-05 18:05:03;2021-12-05 18:05:04;SomeHEXID;SomeIDNumber;Whatever;0;SomethingElse
Each line starts in this way.
I have tried something like
sed -e 's/\(....\)\(..\)\(..\)\(..\)\(..\)\(..\);\(.*\)/\1-\2-\3 \4:\5:\6;\7/' -e 's/\(.*\);\(....\)\(..\)\(..\)\(..\)\(..\)\(..\)/\1;\2-\3-\4 \5:\6:\7/'
and
sed -E 's/(....)(..)(..)(..)(..)(..)(;?)/\1-\2-\3 \4:\5:\6\7/g'
but both of these methods change the text past the first 2 fields.
TIA
There are multiple ways to approach the problem with sed, but all of the best ones will involve using s commands with capturing groups in the pattern and back references in the replacement. However, any solution based on that tool will need to work around the problem that there are too many separate fields (12) in each line of data to associate all of them with separate back references in a single s command. One fairly simple way to accommodate that would be to split it across two s commands, one for the first date code and one for the second. For example:
sed -e 's/\(....\)\(..\)\(..\)\(..\)\(..\)\(..\);\(.*\)/\1-\2-\3 \4:\5:\6;\7/' \
-e 's/\(.*\);\(....\)\(..\)\(..\)\(..\)\(..\)\(..\)/\1;\2-\3-\4 \5:\6:\7/' \
< input > output
The first sed expression matches matches each of the first 14 characters with its own . wildcard, capturing them in separate groups as specified by the escaped parentheses \( ... \), and captures all of the second date code with one additional capturing group matching the tail of the line. It then replaces everything matched (that is, the whole line) with an intermediate form in which the first date code is formatted but the second is not, something like so:
2021-12-05 18:05:03;20211205180504
In the replacement part of that s command, the \1 represents the data captured by the first group, the \2 represents that captured by the second group, etc..
The second sed expression is similar, but it handles the second time code, producing the wanted overall formatting as the end result.
I want to put serialized numbers on defined positions in a text file.
My idea is to use character patterns in the file, count up a variable and put them by using sed in the file. I tried this:
for number in 1 2 3 4 ; do
sed -ibak "s/var/$number" file.txt > file2.txt
done
(the arguments 1 2 3 ... are not the best solution, but I think, it should work)
With this code and tiny variations of it, I get different results, but no success.
I can cut/paste the pattern in the text, but it is always the last argument inserted (="3"). Why doesn´t sed take the iterated variable? (which is counted up, I tested it with echo).
The first iteration replaces var by 1, the next iteration replaces exactly the same var by 2, etc. - because you operate on the same input every time, and the pattern isn't dynamic.
It's not clear what you want to achieve, so it's hard to provide a working solution.
It might be easier to reach for Perl:
perl -pe 's/picvar/"pic" . ++$i/e'
I have a file like this where i need to replace all the strings line with sed command, for example i want to use sed on "Branglebracken" and to replace with other wanted text, but i do not want to replace it when this word is included in other strings. I mean i need to replace it only when the word is only alone, not when it's included in other texts. So basically it must start and end with it...
I came here to Branglebracken with my wife and child, as well as an Altmer herbalist who wanted to explore the forest.\n\nCould you check on them? Since they're not back already, I assume they've taken a rest at the wayshrine to the northwest.
Branglebracken
Branglebracken
Branglebracken
[intentionally repeated]
The output is this
I came here to Branglebracken with my wife and child, as well as an Altmer herbalist who wanted to explore the forest.\n\nCould you check on them? Since they're not back already, I assume they've taken a rest at the wayshrine to the northwest.
word-replaced
word-replaced
word-replaced
[intentionally repeated]
As you notice, the text must still have the word "Branglebracken"
You can use anchors to restrict your match to only that search pattern in a line:
sed 's/^Branglebracken$/word-replaced/' file
^ and $ will ensure you only match Branglebracken in a line.
I know that you can use this to remove blank lines
sed /^$/d
and this to remove comments starting with #
sed /^#/d
but how to you do delete all the comments starting with // ?
You just need to "escape" the slashes with the backslash.
/\/\//
the ^ operator binds it to the front of the line, so your example will only affect comments starting in the first column. You could try adding spaces and tabs in there, too, and then use the alternation operator | to choose between two comment identifiers.
/^[ \t]*(\/\/|$)/
Edit:
If you simply want to remove comments from the file, then you can do something like:
/(\/\/|$).*/
I don't know what the 'd' operator at the end does, but the above expression should match for you modulo having to escape the parentheses or the alternation operator (the '|' character)
Edit 2:
I just realized that using a Mac you may be "shelling" that command and using the system sed. In that case, you could try putting quotation marks around the search pattern so that the shell doesn't do anything crazy to all of your magic characters. :) In this case, 'd' means "delete the pattern space," so just stick a 'd' after the last example I gave and you should be set.
Edit 3:
Oh I just realized, you'll want to beware that if you don't catch things inside of quotes (i.e. you don't want to delete from # to end of line if it's in a string!). The regexp becomes quite a bit more complicated in that case, unfortunately, unless you just forgo checking lines with strings for comments. ...but then you'd need to use the substitution operation to sed rather than search-and-delete-match. ...and you'd need to put in more escapes, and it becomes madness. I suggest searching for an online sed helper (there are good regex testers out there, maybe there's one for sed?).
Sorry to sort of abandon the project at this point. This "problem" is one that sed can do but it becomes substantially more complex at every stage, as opposed to just whipping up a bit of Python to do it.
I have been looking at regular expressions to try and do this, but the most I can do is find the start of a line with ^, but not replace it.
I can then find the first characters on a line to replace, but can not do it in such a way with keeping it intact.
Unfortunately I don´t have access to a tool like cut since I am on a windows machine...so is there any way to do what I want with just regexp?
Use notepad++. It offers a way to record an sequence of actions which then can be repeated for all lines in the file.
Did you try replacing the regular expression ^ with the text you want to put at the start of each line? Also you should use the multiline option (also called m in some regex dialects) if you want ^ to match the start of every line in your input rather than just the first.
string s = "test test\ntest2 test2";
s = Regex.Replace(s, "^", "foo", RegexOptions.Multiline);
Console.WriteLine(s);
Result:
footest test
footest2 test2
I used to program on the mainframe and got used to SPF panels. I was thrilled to find a Windows version of the same editor at Command Technology. Makes problems like this drop-dead simple. You can use expressions to exclude or include lines, then apply transforms on just the excluded or included lines and do so inside of column boundaries. You can even take the contents of one set of lines and overlay the contents of another set of lines entirely or within column boundaries which makes it very easy to generate mass assignments of values to variables and similar tasks. I use Notepad++ for most stuff but keep a copy of SPFSE around for special-purpose editing like this. It's not cheap but once you figure out how to use it, it pays for itself in time saved.