change only some entries in /etc/fstab file in bash - bash

Below is a sample /etc/fstab with 9 entries. How can I change column 4 (from-defaults to-noexec,nodev) for nfs shares only? How can I use an exceptions file to do this? or is there any other way if i want to do this system wide?
/dev/appvg/me /me ext4 defaults 0 0
/dev/appvg/you /you ext4 defaults 0 0
/dev/appvg/we /we ext4 defaults 0 0
/dev/appvg/us /us ext4 defaults 0 0
nas-prd-unix.com:/vol/unix/he /he nfs defaults 0 0
nas-prd-unix.com:/vol/unix/she /she nfs defaults 0 0
nas-prd-unix.com:/vol/unix/it /it nfs defaults 0 0
nas-prd-unix.com:/vol/unix/one /one nfs defaults 0 0
nas-prd-unix.com:/vol/unix/two /two nfs defaults 0 0

In awk:
$ awk '$1 ~ /test[3469]/{sub(/defaults {6}/,"defaluts,nodev",$0)} 1' /etc/fstab
/dev/appvg/test1 /test1 nfs defaults 0 0
/dev/appvg/test2 /test2 nfs defaults 0 0
/dev/appvg/test3 /test3 nfs defaluts,nodev 0 0
/dev/appvg/test4 /test4 nfs defaluts,nodev 0 0
/dev/appvg/test5 /test5 nfs defaults 0 0
/dev/appvg/test6 /test6 nfs defaluts,nodev 0 0
/dev/appvg/test7 /test7 nfs defaults 0 0
/dev/appvg/test8 /test8 nfs defaults 0 0
/dev/appvg/test9 /test9 nfs defaluts,nodev 0 0
If you insist on exclusion file:
$ cat exclusions.txt
/dev/appvg/test1
/dev/appvg/test2
/dev/appvg/test5
/dev/appvg/test7
/dev/appvg/test8
Script:
$ cat script.awk
NR==FNR { a[$1];next } # remember exclusions
!($1 in a) { sub(/defaults {6}/,"defaluts,nodev",$0) } # don't replace on them
1 # print
Run it:
$ awk -f script.awk exclusions.txt /etc/fstab
# for output, see above.

You can use a regular expression with a look-behind assertion. These are supported by Perl, for example:
perl -pe 's/test[3469].*\Kdefaults/defaults,nodev/' /etc/fstab
To change the file in-place with a backup you can do:
perl -p -i.bak -e 's/test[3469].*\Kdefaults/defaults,nodev/' /etc/fstab

Somewhat similar to Eugene's answer...
# Your input data or file "data.txt"
bash$> cat data.txt
/dev/appvg/test1 /test1 nfs defaults 0 0
/dev/appvg/test2 /test2 nfs defaults 0 0
/dev/appvg/test3 /test3 nfs defaults 0 0
/dev/appvg/test4 /test4 nfs defaults 0 0
/dev/appvg/test5 /test5 nfs defaults 0 0
/dev/appvg/test6 /test6 nfs defaults 0 0
/dev/appvg/test7 /test7 nfs defaults 0 0
/dev/appvg/test8 /test8 nfs defaults 0 0
/dev/appvg/test9 /test9 nfs defaults 0 0
bash$>
# Apply a perl command, which performs in place editing.
** Caution: If you don't formulate the command properly, it may damage the file **
bash$> perl -pi -e 'if (m/test[3469]/) {$_ =~ s/defaults/defaults,nodev/g}' data.txt ; cat data.txt
/dev/appvg/test1 /test1 nfs defaults 0 0
/dev/appvg/test2 /test2 nfs defaults 0 0
/dev/appvg/test3 /test3 nfs defaults,nodev 0 0
/dev/appvg/test4 /test4 nfs defaults,nodev 0 0
/dev/appvg/test5 /test5 nfs defaults 0 0
/dev/appvg/test6 /test6 nfs defaults,nodev 0 0
/dev/appvg/test7 /test7 nfs defaults 0 0
/dev/appvg/test8 /test8 nfs defaults 0 0
/dev/appvg/test9 /test9 nfs defaults,nodev 0 0
bash$>
If you do this often enough, then use this script, and just feed parameters to it as shown and it will formulate a command and execute it for you.
# Script output
bash$> ./run.sh "test[3469]" "defaults" "defaults,nodev"
Formed Command is : perl -pi -e 'if(m/test[3469]/) {s/defaults/defaults,nodev/g}' data.txt
/dev/appvg/test1 /test1 nfs defaults 0 0
/dev/appvg/test2 /test2 nfs defaults 0 0
/dev/appvg/test3 /test3 nfs defaults,nodev 0 0
/dev/appvg/test4 /test4 nfs defaults,nodev 0 0
/dev/appvg/test5 /test5 nfs defaults 0 0
/dev/appvg/test6 /test6 nfs defaults,nodev 0 0
/dev/appvg/test7 /test7 nfs defaults 0 0
/dev/appvg/test8 /test8 nfs defaults 0 0
/dev/appvg/test9 /test9 nfs defaults,nodev 0 0
bash$>
Here is the script, which can be made general purpose if one needs to do this frequently, although this is not the only way. One can devise tonnes of other ways to arrive at the same result. This is just one approach.
bash$> cat run.sh
#!/bin/bash
# Taking Input parameters in a,b,c
a="$1"
b="$2"
c="$3"
# Old command for reference
#perl -pi -e 'if (m/test[3469]/) {$_ =~ s/defaults/defaults,nodev/g}' data.txt ; cat data.txt
# Now formulate a command and use substitution
cmd="perl -pi -e 'if(m/$a/) {s/$b/$c/g}' data.txt"
echo "Formed Command is : $cmd"
# Execute the formulated command
eval "$cmd"
# Check the output after execution.
echo ; cat data.txt
bash$>

Related

Is there any specific way to extract SSD name without "/"?

In my current system, I have 3 SSDs, sda, sdb and sdc. The OS is installed in sdc.
I am trying to extract the SSDs without the OS installed in it. So, this command
echo $(eval $(lsblk -oMOUNTPOINT,PKNAME -P | grep 'MOUNTPOINT="/"'); echo $PKNAME | sed 's/[0-9]*$//')
returns sdc.
But if I want the drive without OS, how should I modify the above command?
grep 'MOUNTPOINT!="/"' doesn't return anything.
The bash script provided by Renaud works as expected on systems with raw drives. In case, if the system has LVM partitions then it returns only dm-1.
How to handle this case? To get the correct SSD name on either raw or LVM systems?
On the LVM system, lsblk returns. The expected output is sdb
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 1.8T 0 disk
├─sda1 8:1 0 512M 0 part /boot/efi
├─sda2 8:2 0 1G 0 part /boot
└─sda3 8:3 0 1.8T 0 part
├─ubuntu--vg-ubuntu--lv-real
│ 253:1 0 880G 0 lvm
│ ├─ubuntu--vg-ubuntu--lv 253:2 0 880G 0 lvm /
│ └─ubuntu--vg-clean 253:4 0 880G 0 lvm
└─ubuntu--vg-clean-cow 253:3 0 400G 0 lvm
└─ubuntu--vg-clean 253:4 0 880G 0 lvm
sdb 8:16 0 1.8T 0 disk
On the raw drive, lsblk returns
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 223.6G 0 disk
sdb 8:16 0 465.8G 0 disk
├─sdb1 8:17 0 2G 0 part
└─sdb2 8:18 0 463.8G 0 part
sdc 8:32 0 232.9G 0 disk
├─sdc1 8:33 0 156M 0 part /boot/efi
├─sdc2 8:34 0 26.7G 0 part /boot
├─sdc3 8:35 0 182.7G 0 part /
└─sdc4 8:36 0 23.4G 0 part [SWAP]
Here the expected output is sda or sdc.
grep has a -v option to print only non-matching lines. So:
lsblk -oMOUNTPOINT,PKNAME -P | grep -v 'MOUNTPOINT="/"'
should exclude this drive. But note that if you have more than one other drive the rest of your script will not work as you would like. Only the last drive will be considered because it overrides the others. A loop would probably be closer to your needs:
while IFS= read -r line; do
eval "$line"
sed 's/[0-9]*$//' <<< "$PKNAME"
done < <(lsblk -oMOUNTPOINT,PKNAME -P | grep -v 'MOUNTPOINT="/"')

If elif else not working in bash ChromeOS

I am trying to make a bash script which basically takes a bunch of .debs, unpacks them and place binaries and libs in /usr/local/opt/{lib}bin.
The script checks whether / is mounted as ro or rw, and if mounted as ro to remount it as rw.
On chromebooks however, in order to mount / as rw you need to remove_rootfs_verification for the partition in question. The script fails to echo what stated above when rootfs_verification is enabled for /, and should exit 1, instead it carries on.
Here is the part of the script I am referring to
### ChromeOS's Specific!!!
# The following assumes rootfs_verification for / has already been removed
if grep $rootfs /proc/mounts | grep ro; then
mount -o remount,rw / &> mount.out
elif
grep -iw 'read-write' mount.out; then
echo '\nrootfs_verification for the root partition must to be removed in order to remount,rw /
To remove rootfs_verification run the following command and than reboot the system:
"sudo /usr/share/vboot/bin/make_dev_ssd.sh --remove_rootfs_verification --partitions 4"'
else exit 1
fi
The entire WIP script can be found here https://pastebin.com/ekEPSvYy
This is what happen when I execute it
localhost /usr/local # ./kvm_install.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 124 0 124 0 0 322 0 --:--:-- --:--:-- --:--:-- 345
100 135 100 135 0 0 170 0 --:--:-- --:--:-- --:--:-- 170
100 60384 100 60384 0 0 57950 0 0:00:01 0:00:01 --:--:-- 344k
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 143 0 143 0 0 407 0 --:--:-- --:--:-- --:--:-- 412
100 154 100 154 0 0 202 0 --:--:-- --:--:-- --:--:-- 202
100 1298k 100 1298k 0 0 929k 0 0:00:01 0:00:01 --:--:-- 3020k
/dev/root / ext2 ro,seclabel,relatime,block_validity,barrier,user_xattr,acl 0 0
./kvm_install.sh: line 31: /etc/env.d/30kvm: Read-only file system
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 66802 100 66802 0 0 69657 0 --:--:-- --:--:-- --:--:-- 74555
./kvm_install.sh: line 39: ar: command not found
tar (child): control.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
md5sum: md5sums: No such file or directory
Basically what happens here is ar cannot be found as the script was unable to add the PATH variables to /etc/env.d/30kvm since the root partition cannot be mounted because roots_verification is enabled on /.
I tried adding the elif "grep" command in [[ as some suggested here, but that didn't work and adds further syntax issues.
I am in the process of learnign the basics of bash scripting. I apologize if the script is written poorly.
Thanks
I ultimately ended up doing this.
if grep $rootfs /proc/mounts | grep 'ro,'; then
mount -o remount,rw / &> mount.out
if
grep 'read-write' mount.out; then
echo 'something to echo' && exit 1
fi
fi
It is not pretty, but it works until I find/ learn a better way to implement the loop.
to make the output of a command a variable do:
$variable="$(command)"
if you want to use grep use the syntax:
command | grep text
if you want to do if statements use the syntax:
if [ some text ]; text
commands
elif [ some text ]
commands
else
commands
fi
for the some text check this cart
grep -iw is not valid on chromebook
rootfs has a different path depending on a lot of things if you want to save it as $rootfs do this command
rootfs="$(rootdev -s)"
also you made some mistake is the "|" and "&"
command1 | command to edit command1
command1 || run this command if command1 fails
command1 && run this command if command1 succeeds

How to get bash to print the output without the fields with zero size when running smem command?

Here is the 'smem' command I run on the Redhat/CentOS Linux system. I expect the output be printed without the fields with zero size however I would expect the heading columns.
smem -kt -c "pid user command swap"
PID User Command Swap
7894 root /sbin/agetty --noclear tty1 0
9666 root ./nimbus /opt/nimsoft 0
7850 root /sbin/auditd 236.0K
7885 root /usr/sbin/irqbalance --fore 0
11205 root nimbus(hdb) 0
10701 root nimbus(spooler) 0
8446 trapsanalyzer1 /opt/traps/analyzerd/analyz 0
50316 apache /usr/sbin/httpd -DFOREGROUN 0
50310 apache /usr/sbin/httpd -DFOREGROUN 0
3971 root /usr/sbin/lvmetad -f 36.0K
63988 root su - 0
7905 ntp /usr/sbin/ntpd -u ntp:ntp - 4.0K
7876 dbus /usr/bin/dbus-daemon --syst 44.0K
9672 root nimbus(controller) 0
7888 root /usr/lib/systemd/systemd-lo 0
63990 root -bash 0
59978 postfix pickup -l -t unix -u 0
3977 root /usr/lib/systemd/systemd-ud 736.0K
9016 postfix qmgr -l -t unix -u 0
50303 root /usr/sbin/httpd -DFOREGROUN 0
3941 root /usr/lib/systemd/systemd-jo 52.0K
8199 root //usr/lib/vmware-caf/pme/bi 0
8598 daemon /opt/quest/sbin/.vasd -p /v 0
8131 root /usr/sbin/vmtoolsd 0
7881 root /usr/sbin/NetworkManager -- 8.0K
8364 root /opt/puppetlabs/puppet/bin/ 0
8616 daemon /opt/quest/sbin/.vasd -p /v 0
23290 root /usr/sbin/rsyslogd -n 3.8M
64091 root python /bin/smem -kt -c pid 0
7887 polkitd /usr/lib/polkit-1/polkitd - 0
8363 root /usr/bin/python2 -Es /usr/s 0
53606 root /usr/share/metricbeat/bin/m 0
24631 nagios /usr/local/ncpa/ncpa_passiv 0
24582 nagios /usr/local/ncpa/ncpa_listen 0
7886 root /opt/traps/bin/authorized 76.0K
7872 root /opt/traps/bin/pmd 12.0K
8374 root /opt/puppetlabs/puppet/bin/ 0
7883 root /opt/traps/bin/trapsd 64.0K
----------------------------------------------------
54 10 5.1M
Like this?:
$ awk '$NF!=0' file
PID User Command Swap
7850 root /sbin/auditd 236.0K
...
7883 root /opt/traps/bin/trapsd 64.0K
----------------------------------------------------
54 10 5.1M
But instead of using the form awk ... file you'd probably like to smem ... | awk '$NF!=0'.
Could you please try following, for extra precautions removing the space from last fields(in case it is there).
smem -kt -c "pid user command swap" | awk 'FNR==1{print;next} {sub(/[[:space:]]+$/,"")} $NF==0{next} 1'

how can I append an entry to a fstab next to a last line entry?

Hi Im writing a shell script that will append below entry to fstab on multiple server.
/dev/vg00/lv_berf /opt/berf ext3 defaults 1 2
/dev/vg00/lv_vberf /var/opt/berf ext3 defaults 1 2
But some fstab has different entry example
fstab on server1:
LABEL=/boot /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/vg00/lv_swap swap swap defaults 0 0
########### ABCD specific FS ############################################
/dev/vgabdc/lv_apal3 /opt/apal3 ext3 defaults 1 2
fstab on server 2
/
dev/vgabc/lv_vabc /var/opt/das ext3 defaults 1 2
/dev/vgabc/lv_vabcoracle /var/opt/oracle/abc ext3 defaults 1 2
/dev/vgabc/lv_vgdb20u01 /var/opt/vgdb20 ext3 defaults 1 2
/dev/vgabc/lv_dasredprairie /opt/redprairie ext3 defaults 1 2
/dev/vgabc/lv_common /opt/common ext3 defaults 1 2
########### ABCD Failover filesystems ############################################
/dev/vgabcd/lv_apal3 /abcd/opt/apal3 ext3 defaults 1 2
I want it to be appended before "########### ABCD ..." and next to the last entry
fstab on server 1
LABEL=/boot /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/vg00/lv_swap swap swap defaults 0 0
/dev/vg00/lv_berf /opt/berf ext3 defaults 1 2
/dev/vg00/lv_vberf /var/opt/berf ext3 defaults 1 2
########### ABCD specific FS ############################################
/dev/vgabdc/lv_apal3 /opt/apal3 ext3 defaults 1 2
fstab on server 2
/dev/vgabc/lv_vabc /var/opt/das ext3 defaults 1 2
/dev/vgabc/lv_vabcoracle /var/opt/oracle/abc ext3 defaults 1 2
/dev/vgabc/lv_vgdb20u01 /var/opt/vgdb20 ext3 defaults 1 2
/dev/vgabc/lv_dasredprairie /opt/redprairie ext3 defaults 1 2
/dev/vgabc/lv_common /opt/common ext3 defaults 1 2
/dev/vg00/lv_berf /opt/berf ext3 defaults 1 2
/dev/vg00/lv_vberf /var/opt/berf ext3 defaults 1 2
########### ABCD Failover filesystems ############################################
/dev/vgabcd/lv_apal3 /abcd/opt/apal3 ext3 defaults 1 2
I have a working script using sed
line_count=`ssh $i grep -n "ABCD" fstab | cut -d: -f1`
next_line=$((line_count + 1))
ssh $i sed -i "${line_count}i\ '/dev/vg00/lv_berf /opt/berf ext3 defaults 1 2'" fstab
ssh $i sed -i "${next_line}i\ '/dev/vg00/lv_vberf /var/opt/berf ext3 defaults 1 2'" fstab
Problem is when the fstab entry has a space before ABCD it will append next to the space. like the example on fstab on server1 it has space before ###ABCD so when the script run it will look below:
tab1
LABEL=/boot /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/vg00/lv_swap swap swap defaults 0 0
/dev/vg00/lv_berf /opt/berf ext3 defaults 1 2
/dev/vg00/lv_vberf /var/opt/berf ext3 defaults 1 2
########### ABCD specific FS ############################################
/dev/vgabdc/lv_apal3 /opt/apal3 ext3 defaults 1 2
How can a fix that with out affecting the other entry on fstab on server 2 which has different last entry?
use function addmntent()
see man page of addmntent

Mac OS X: GNU parallel can't find the number of cores on a remote server

I used homebrew to install GNU parallel on my mac so I can run some tests remotely on my University's servers. I was quickly running through the tutorials, but when I ran
parallel -S <username>#$SERVER1 echo running on ::: <username>#$SERVER1
I got the message
parallel: Warning: Could not figure out number of cpus on <username#server> (). Using 1.
Possibly related, I never added parallel to my path and got the warning that "parallel" wasn't a recognized command, but parallel ran anyways and still echo'd correctly. This particular server has 16 cores, how can I get parallel to recognize them?
GNU Parallel is less tested on OS X as I do not have access to an OS X installation, so you have likely found a bug.
GNU Parallel has since 20120322 used these to find the number of CPUs:
sysctl -n hw.physicalcpu
sysctl -a hw 2>/dev/null | grep [^a-z]physicalcpu[^a-z] | awk '{ print \$2 }'
And the number of cores:
sysctl -n hw.logicalcpu
sysctl -a hw 2>/dev/null | grep [^a-z]logicalcpu[^a-z] | awk '{ print \$2 }'
Can you test what output you get from those?
Which version of GNU Parallel are you using?
As a work around you can force GNU Parallel to detect 16 cores:
parallel -S 16/<username>#$SERVER1 echo running on ::: <username>#$SERVER1
Since version 20140422 you have been able to export your path to the remote server:
parallel --env PATH -S 16/<username>#$SERVER1 echo running on ::: <username>#$SERVER1
That way you just need to add the dir where parallel lives on the server to your path on local machine. E.g. parallel on the remote server is in /home/u/user/bin/parallel:
PATH=$PATH:/home/u/user/bin parallel --env PATH -S <username>#$SERVER1 echo running on ::: <username>#$SERVER1
Information for Ole
My iMac (OSX MAvericks on Intel core i7) gives the following, which all looks correct:
sysctl -n hw.physicalcpu
4
sysctl -a hw
hw.ncpu: 8
hw.byteorder: 1234
hw.memsize: 17179869184
hw.activecpu: 8
hw.physicalcpu: 4
hw.physicalcpu_max: 4
hw.logicalcpu: 8
hw.logicalcpu_max: 8
hw.cputype: 7
hw.cpusubtype: 4
hw.cpu64bit_capable: 1
hw.cpufamily: 1418770316
hw.cacheconfig: 8 2 2 8 0 0 0 0 0 0
hw.cachesize: 17179869184 32768 262144 8388608 0 0 0 0 0 0
hw.pagesize: 4096
hw.busfrequency: 100000000
hw.busfrequency_min: 100000000
hw.busfrequency_max: 100000000
hw.cpufrequency: 3400000000
hw.cpufrequency_min: 3400000000
hw.cpufrequency_max: 3400000000
hw.cachelinesize: 64
hw.l1icachesize: 32768
hw.l1dcachesize: 32768
hw.l2cachesize: 262144
hw.l3cachesize: 8388608
hw.tbfrequency: 1000000000
hw.packages: 1
hw.optional.floatingpoint: 1
hw.optional.mmx: 1
hw.optional.sse: 1
hw.optional.sse2: 1
hw.optional.sse3: 1
hw.optional.supplementalsse3: 1
hw.optional.sse4_1: 1
hw.optional.sse4_2: 1
hw.optional.x86_64: 1
hw.optional.aes: 1
hw.optional.avx1_0: 1
hw.optional.rdrand: 0
hw.optional.f16c: 0
hw.optional.enfstrg: 0
hw.optional.fma: 0
hw.optional.avx2_0: 0
hw.optional.bmi1: 0
hw.optional.bmi2: 0
hw.optional.rtm: 0
hw.optional.hle: 0
hw.cputhreadtype: 1
hw.machine = x86_64
hw.model = iMac12,2
hw.ncpu = 8
hw.byteorder = 1234
hw.physmem = 2147483648
hw.usermem = 521064448
hw.pagesize = 4096
hw.epoch = 0
hw.vectorunit = 1
hw.busfrequency = 100000000
hw.cpufrequency = 3400000000
hw.cachelinesize = 64
hw.l1icachesize = 32768
hw.l1dcachesize = 32768
hw.l2settings = 1
hw.l2cachesize = 262144
hw.l3settings = 1
hw.l3cachesize = 8388608
hw.tbfrequency = 1000000000
hw.memsize = 17179869184
hw.availcpu = 8
sysctl -n hw.logicalcpu
8

Resources