I am trying to create partition using the below script. It works, but sometimes I need to re-run the same script due to some automation stuff during the time I am getting error like already in use.
parted /dev/sdc --script mklabel gpt mkpart xfspart xfs 0% 100%
mkfs.xfs /dev/sdc1
mkdir -p /opt/app
lsblk
echo "/dev/sdc1 /opt/app xfs defaults 0 1" >>/etc/fstab
mount -a
df -Th
Error log when I executing the same script again.
+ parted /dev/sdc --script mklabel gpt mkpart xfspart xfs 0% 100%
Error: Partition(s) on /dev/sdc are being used.
+ mkfs.xfs /dev/sdc1
mkfs.xfs: /dev/sdc1 contains a mounted filesystem
How can I skip if the file system from /dev/sdc is already mounted?
You can check if the mount is already there, and skip the script if it's found;
if grep -qs '/dev/sdc' /proc/mounts; then
echo "Skipping mount since /dev/sdc already exists"
else
parted /dev/sdc --script mklabel gpt mkpart xfspart xfs 0% 100%
mkfs.xfs /dev/sdc1
mkdir -p /opt/app
lsblk
echo "/dev/sdc1 /opt/app xfs defaults 0 1" >>/etc/fstab
mount -a
df -Th
fi
Related
After adding volumes to an ec2 instance using ansible. How can I mount these devices by size to desired mount point using a shell script that I will pass to user_data
[ec2-user#xxx ~]$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259:4 0 200G 0 disk
├─nvme0n1p1 259:5 0 1M 0 part
└─nvme0n1p2 259:6 0 200G 0 part /
nvme1n1 259:0 0 70G 0 disk
nvme2n1 259:1 0 70G 0 disk
nvme3n1 259:3 0 70G 0 disk
nvme4n1 259:2 0 20G 0 disk
This is what i wrote initially but realized the NAME and SIZE are not same always for nvm's
#!/bin/bash
VOLUMES=(nvme1n1 nvme2n1 nvme3n1 nvme4n1)
PATHS=(/abc/sfw /abc/hadoop /abc/log /kafka/data/sda)
for index in ${!VOLUMES[*]}; do
sudo mkfs -t xfs /dev/"${VOLUMES[$index]}"
sudo mkdir -p "${PATHS[$index]}"
sudo mount /dev/"${VOLUMES[$index]}" "${PATHS[$index]}"
echo "Mounted ${VOLUMES[$index]} in ${PATHS[$index]}"
done
I am creating these using ansible and want the 20G to be mounted on /edw/logs but 20G randomly goes on any device. (nvme1n1 or nvme2n1 or nvme3n1 or nvme4n1)
How to write/modify my script?
I have installed ubuntu touch on my nexus 7 2013 recently, and have faced problem installing driver for Moxa Uport usb adapter. Actually the driver needs to be compilled and this is why I need linux headers to be installed.
I have found, that:
apt-get install linux-headers-`uname -r`
needs to write something to /lib/modules and this path belongs to /dev/loop1 which is mounted as readonly. I'm able to remount it to rw, but on single attempt to write there, it falls back to readonly.
nothing about loop1 or /lib/modules in /etc/fstab
command:
mount | grep loop1
output:
/dev/loop1 on /android/system type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /etc/ubuntu-touch-session.d/android.conf type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /lib/udev/rules.d/70-android.rules type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /usr/share/powerd/device_configs/config-default.xml type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /lib/modules type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /var/lib/lxc/android/rootfs/system type ext4 (ro,relatime,data=ordered)
command:
blockdev --report |grep loop1
output:
ro 256 512 4096 0 126427136 /dev/loop1
then:
blockdev --setrw /dev/loop1
blockdev --report |grep loop1
output:
rw 256 512 4096 0 126427136 /dev/loop1
command:
mount -o remount,rw /dev/loop1
mount | grep loop1
output:
/dev/loop1 on /android/system type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /etc/ubuntu-touch-session.d/android.conf type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /lib/udev/rules.d/70-android.rules type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /usr/share/powerd/device_configs/config-default.xml type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /lib/modules type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /var/lib/lxc/android/rootfs/system type ext4 (rw,relatime,data=ordered)
strange, only /var/lib/lxc/android/rootfs/system was set to rw
so:
mount -o remount,rw /lib/modules
mount | grep loop1
output:
/dev/loop1 on /android/system type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /etc/ubuntu-touch-session.d/android.conf type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /lib/udev/rules.d/70-android.rules type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /usr/share/powerd/device_configs/config-default.xml type ext4 (ro,relatime,data=ordered)
/dev/loop1 on /lib/modules type ext4 (rw,relatime,data=ordered)
/dev/loop1 on /var/lib/lxc/android/rootfs/system type ext4 (rw,relatime,data=ordered)
but:
mkdir /lib/modules/rrr
output:
mkdir: cannot create directory Б─≤/lib/modules/rrrБ─≥: Read-only file system
dmesg:
[ 8280.681213] EXT4-fs (loop1): previous I/O error to superblock detected
[ 8280.681579] Buffer I/O error on device loop1, logical block 0
[ 8280.682006] lost page write due to I/O error on loop1
[ 8280.682037] EXT4-fs error (device loop1): ext4_journal_start_sb:328: Detected aborted journal
[ 8280.682891] EXT4-fs (loop1): Remounting filesystem read-only
[ 8280.683135] EXT4-fs (loop1): previous I/O error to superblock detected
[ 8280.683593] Buffer I/O error on device loop1, logical block 0
I think the abowe is the reason of:
apt-get install linux-headers-`uname -r`
fails with:
Unpacking linux-headers-3.4.0-5-flo (3.4.0-5.19~15.04.1) ...
dpkg: error processing archive /var/cache/apt/archives/linux-headers-3.4.0-5-flo_3.4.0-5.19~15.04.1_armhf.deb (--unpack):
error creating symbolic link `./lib/modules/3.4.0-5-flo/build': Read-only file system
Errors were encountered while processing:
/var/cache/apt/archives/linux-headers-3.4.0-5-flo_3.4.0-5.19~15.04.1_armhf.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
So, I got stuck on this, could please anybody help me?
Regards, Victor
Well, being not very experienced in ubuntu touch, android, linux, armhf stuff, I wasn't able to do it right and straightforward way.
Perhaps hacking boot.img could help but I'm not ready for this.
The only workaround I was able to imagine is mounting /lib/modules/3.4.0-5-flo to different new loop device. It works though.
cd /userdata
dd bs=1M count=100 if=/dev/zero of=build.img
losetup /dev/loop2 build.img
mkfs -t ext3 -m 1 -v /dev/loop2
mkdir /eee
mount -t ext3 /dev/loop2 /eee
cp -r /lib/modules/3.4.0-5-flo/. /eee
umount /eee
mount /dev/loop2 /lib/modules/3.4.0-5-flo
rmdir /eee
From this point I got able to install linux headers and compile Moxa's drivers.
I have discovered that drivers makefile aslo writes to /lib/modules/3.4.0-5-flo by the way.
So, to make mount permanent, and failed to change /etc/fstab I had to add to rc.local:
losetup /dev/loop2 /userdata/build.img
mount /dev/loop2 /lib/modules/3.4.0-5-flo
modprobe mxu11x0
I know it's rather wierd solution, but worked for me :)
on ubuntu touch the rootfs is read only so you would have to remount that as read write in general to be able to use apt on it.
Just starting on chef, I'm trying to convert my old bash provisioning scripts to something more modern and realiable, using Chef.
The first script is somewhat i used to build a partition and mount it to /opt.
This is the script: https://github.com/theclue/db2-vagrant/blob/master/provision_for_mount_disk.sh
#!/bin/bash
yum install -y parted
parted /dev/sdb mklabel msdos
parted /dev/sdb mkpart primary ext4 0% 100%
sleep 3
#-m swith tells mkfs to only reserve 1% of the blocks for the super block
mkfs.ext4 /dev/sdb1
e2label /dev/sdb1 "opt"
######### mount sdb1 to /opt ##############
chmod 777 /opt
mount /dev/sdb1 /opt
chmod 777 /opt
echo '/dev/sdb1 /opt ext4 defaults 0 0' >> /etc/fstab
I found a parted recipe here, but it doesn't seem to support all the parameters I need (0% and 100%, to name two) and anyway I've no idea on how to do the formatting/mounting block.
any idea?
I've been working on a simple fully-automated installer for ArchLinux recently.
I started with a basic three-partition system; that made a bootable system.
I added LVM and several more logical partitions; that worked also.
I'm now trying to add LUKS encryption; that isn't working.
I want to have LUKS inside of LVM for more flexibility. For a BIOS system with a single disk, that should look something like this:
Raw Partitions
+-----------+------+-------+-----------+
| Partition | Name | Size | Flags |
+-----------+------+-------+-----------+
| /dev/sda1 | grub | 2MB | bios_grub |
| /dev/sda2 | boot | 200MB | boot |
| /dev/sda3 | lvm | | lvm |
+-----------+------+-------+-----------+
LVM Partitions
+-------------+--------------+-------+------+
| LVM Device | LUKS Device | Name | Size |
+-------------+--------------+-------+------+
| LvmDvc-root | LuksDvc-root | root | 2GB |
| LvmDvc-home | LuksDvc-home | home | 2GB |
| LvmDvc-var | LuksDvc-var | var | 1G |
| LvmDvc-usr | LuksDvc-usr | usr | 1G |
| LvmDvc-swap | LuksDvc-swap | swap | 4G |
+-------------+--------------+-------+------+
LvmDvc-root is decrypted to LuksDvc-root using a passphrase.
All other LVM devices are decrypted using keys stored in /etc/ctyptkeys.
Partitions are mounted as:
/dev/mapper/LuksDvc-root -> /
/dev/sda2 -> /boot
/dev/mapper/LuksDvc-home -> /home
/dev/mapper/LuksDvc-var -> /var
/dev/mapper/LuksDvc-usr -> /usr
From what I can tell, disk partitioning and system install work just fine. I receive a slew of errors about lvmetad not being loaded during grub configuration, but the documentation in Arch's wiki indicates this is a non-issue (https://wiki.archlinux.org/index.php/GRUB#UEFI_systems_2). I also receive these same errors in the previous version of my script (using LVM, but not LUKS), and it produces a bootable system. So I don't think this error message indicates a problem.
When I boot the system, I get through GRUB just fine. I am presented with a dialogue to decrypt the root partition:
A password is required to access the LuksDvc-root volume:
Enter passphrase for /dev/mapper/LvmDvc-root:
I enter the passphrase used during installation, and receive this message:
No key available with this passphrase.
I'm using a very simple passphrase for testing (asdfasdf), so I doubt I'm messing it up. I can decrypt and mount the whole system from the live installer without incident, I just can't make it happen at boot.
I'm not sure what information would be most helpful for solving this. Here is the script I use to install the system:
#!/usr/bin/env bash
set -ex -o pipefail -o nounset
# Raw Partitioning
parted --script --align optimal -- /dev/sda mklabel gpt
parted --script --align optimal -- /dev/sda mkpart primary 2 4
parted --script --align optimal -- /dev/sda name 1 bios_grub
parted --script --align optimal -- /dev/sda set 1 bios_grub on
parted --script --align optimal -- /dev/sda mkpart primary 4 204
parted --script --align optimal -- /dev/sda name 2 boot
parted --script --align optimal -- /dev/sda set 2 boot on
parted --script --align optimal -- /dev/sda mkpart primary 204 -1
parted --script --align optimal -- /dev/sda name 3 lvm
parted --script --align optimal -- /dev/sda set 3 lvm on
# LVM Partitioning
pvcreate -ff --yes /dev/sda3
vgcreate LvmDvc /dev/sda3
lvcreate --zero y --wipesignatures y --name root --size 2G LvmDvc
lvcreate --zero y --wipesignatures y --name home --size 2G LvmDvc
lvcreate --zero y --wipesignatures y --name var --size 1G LvmDvc
lvcreate --zero y --wipesignatures y --name usr --size 1G LvmDvc
lvcreate --zero y --wipesignatures y --name swap --size 4G LvmDvc
# Root Partition
echo asdfasdf | cryptsetup -q --key-file - luksFormat /dev/mapper/LvmDvc-root
echo asdfasdf | cryptsetup -q --key-file - luksOpen /dev/mapper/LvmDvc-root LuksDvc-root
mkfs.ext4 -q /dev/mapper/LuksDvc-root
mkdir -p /mnt/archbox
mount /dev/mapper/LuksDvc-root /mnt/archbox
# Boot Partition
mkfs.ext4 -q /dev/sda2
# Encrypted Partitions
mkdir -p /mnt/archbox/etc/cryptkeys
chmod 400 /mnt/archbox/etc/cryptkeys
dd if=/dev/random of=/mnt/archbox/etc/cryptkeys/home bs=512 count=4 iflag=fullblock
chmod 400 /mnt/archbox/etc/cryptkeys/home
cryptsetup -q --key-file /mnt/archbox/etc/cryptkeys/home luksFormat /dev/mapper/LvmDvc-home
cryptsetup -q --key-file /mnt/archbox/etc/cryptkeys/home luksOpen /dev/mapper/LvmDvc-home LuksDvc-home
mkfs.ext4 -q /dev/mapper/LuksDvc-home
dd if=/dev/random of=/mnt/archbox/etc/cryptkeys/var bs=512 count=4 iflag=fullblock
chmod 400 /mnt/archbox/etc/cryptkeys/var
cryptsetup -q --key-file /mnt/archbox/etc/cryptkeys/var luksFormat /dev/mapper/LvmDvc-var
cryptsetup -q --key-file /mnt/archbox/etc/cryptkeys/var luksOpen /dev/mapper/LvmDvc-var LuksDvc-var
mkfs.ext4 -q /dev/mapper/LuksDvc-var
dd if=/dev/random of=/mnt/archbox/etc/cryptkeys/usr bs=512 count=4 iflag=fullblock
chmod 400 /mnt/archbox/etc/cryptkeys/usr
cryptsetup -q --key-file /mnt/archbox/etc/cryptkeys/usr luksFormat /dev/mapper/LvmDvc-usr
cryptsetup -q --key-file /mnt/archbox/etc/cryptkeys/usr luksOpen /dev/mapper/LvmDvc-usr LuksDvc-usr
mkfs.ext4 -q /dev/mapper/LuksDvc-usr
dd if=/dev/random of=/mnt/archbox/etc/cryptkeys/swap bs=512 count=4 iflag=fullblock
chmod 400 /mnt/archbox/etc/cryptkeys/swap
cryptsetup -q --key-file /mnt/archbox/etc/cryptkeys/swap luksFormat /dev/mapper/LvmDvc-swap
cryptsetup -q --key-file /mnt/archbox/etc/cryptkeys/swap luksOpen /dev/mapper/LvmDvc-swap LuksDvc-swap
mkswap /dev/mapper/LuksDvc-swap
# Mount
mkdir -p /mnt/archbox/boot
mount /dev/sda2 /mnt/archbox/boot
mkdir -p /mnt/archbox/home
mount /dev/mapper/LuksDvc-home /mnt/archbox/home
mkdir -p /mnt/archbox/var
mount /dev/mapper/LuksDvc-var /mnt/archbox/var
mkdir -p /mnt/archbox/usr
mount /dev/mapper/LuksDvc-usr /mnt/archbox/usr
swapon /dev/mapper/LuksDvc-swap
# Packages
mkdir -p ./cache-dir
rm -f /mnt/archbox/var/lib/pacman/db.lck
pacstrap /mnt/archbox --cachedir ./cache-dir base grub
# Root password
echo "root:asdfasdf" | chpasswd --root /mnt/archbox
# FSTab
genfstab -U -p /mnt/archbox >> /mnt/archbox/etc/fstab
# CryptTab
echo "" > /mnt/archbox/etc/crypttab
echo "home /dev/mapper/LvmDvc-home /mnt/archbox/etc/cryptkeys/home" >> /mnt/archbox/etc/crypttab
echo "usr /dev/mapper/LvmDvc-usr /mnt/archbox/etc/cryptkeys/usr" >> /mnt/archbox/etc/crypttab
echo "var /dev/mapper/LvmDvc-var /mnt/archbox/etc/cryptkeys/var" >> /mnt/archbox/etc/crypttab
echo "swap /dev/mapper/LvmDvc-swap /mnt/archbox/etc/cryptkeys/swap" >> /mnt/archbox/etc/crypttab
# Ramdisk
file=/mnt/archbox/etc/mkinitcpio.conf
search="^\s*MODULES=.*$"
replace="MODULES=\\\"virtio virtio_blk virtio_pci virtio_net\\\""
grep -q "$search" "$file" && sed -i "s#$search#$replace#" "$file" || echo "$replace" >> "$file"
search="^\s*HOOKS=.*$"
replace="HOOKS=\\\"base udev autodetect modconf block keymap encrypt lvm2 filesystems keyboard shutdown fsck usr\\\""
grep -q "$search" "$file" && sed -i "s#$search#$replace#" "$file" || echo "$replace" >> "$file"
arch-chroot /mnt/archbox mkinitcpio -p linux
# Bootloader
arch-chroot /mnt/archbox grub-install --target=i386-pc --recheck /dev/sda
file=/mnt/archbox/etc/default/grub
search="^\s*GRUB_CMDLINE_LINUX=.*$"
replace="GRUB_CMDLINE_LINUX=\\\"init=/usr/lib/systemd/systemd cryptdevice=/dev/mapper/LvmDvc-root:LuksDvc-root root=/dev/mapper/LuksDvc-root quiet\\\""
grep -q "$search" "$file" && sed -i "s#$search#$replace#" "$file" || echo "$replace" >> "$file"
search="^\s*GRUB_DISABLE_LINUX_UUID=.*$"
replace="GRUB_DISABLE_LINUX_UUID=true"
grep -q "$search" "$file" && sed -i "s#$search#$replace#" "$file" || echo "$replace" >> "$file"
arch-chroot /mnt/archbox grub-mkconfig -o /boot/grub/grub.cfg
Does anything stick out as blatantly wrong? What should I be doing differently? Can I provide additional/specific information?
tl;dr - Install script seems to work, but I can't decrypt the system at boot. Halp!
I found the main problem: formatting the root LUKS volume should be done with a password, not a key file. Taking out the option --key-file - on the formatting and opening for the root volume fixes the problem I was seeing.
There are some other issues, in the script, but I'll come back after I've tried and failed to fix them.
appreciate any updates of your script; I'm currently testing it. Note, in the above script, the isolated '-' is a syntax error:
echo asdfasdf | cryptsetup -q --key-file - luksFormat /dev/mapper/LvmDvc-root
echo asdfasdf | cryptsetup -q --key-file - luksOpen /dev/mapper/LvmDvc-root LuksDvc-root
after taking out '--key-file' as you suggested, the two lines should be:
echo asdfasdf | cryptsetup -q luksFormat /dev/mapper/LvmDvc-root
echo asdfasdf | cryptsetup -q luksOpen /dev/mapper/LvmDvc-root LuksDvc-root
after this changing the script did run through with the warnings you mentioned. when booting the disk generated by the script, root volume password and then
dev/mapper/Lucks-Dvc-root: clean, ...
ERROR: device '' not found. Skipping fsck.
mount: wrong fs type, bad option, bad superblock ,
missing code page or helper program, ... try dmesg | tail
ERROR: root device mounted success..., but /usr/lib/systemd/systemd does not exist.
Bailing out...
sh: cant access tty: job control turned off
[rootfs /]#
when searching for this error, found:
"Looks like that script doesn't install systemd-sysvcompat, which provides the /sbin/init symlink to /usr/lib/systemd/systemd. You might want to file a bug report and/or use pacstrap in the future."
-- https://bbs.archlinux.org/viewtopic.php?id=146712
(post by WorMzy 2013-02-06 15:11:34)
What about using LVM on top of LUKS instead of the other way around used in your script?
I used the following instructions on a number of machines and it worked just fine: http://is.gd/OoDx1d
I am new to vagrant. I am following the instructions at http://gettingstartedwithdjango.com/en/lessons/introduction-and-launch/
I am getting the following error on running "sudo ./postinstall.sh" script
+ apt-get -y clean
+ rm -f /var/lib/dhcp3/*
+ rm /etc/udev/rules.d/70-persistent-net.rules
rm: cannot remove `/etc/udev/rules.d/70-persistent-net.rules': Is a directory
+ mkdir /etc/udev/rules.d/70-persistent-net.rules
mkdir: cannot create directory `/etc/udev/rules.d/70-persistent-net.rules': File exists
+ rm -rf /dev/.udev/
+ rm /lib/udev/rules.d/75-persistent-net-generator.rules
rm: cannot remove `/lib/udev/rules.d/75-persistent-net-generator.rules': No such file or directory
+ rm -f /home/vagrant/{*.iso,postinstall*.sh}
+ dd if=/dev/zero of=/EMPTY bs=1M
dd: writing `/EMPTY': No space left on device
78504+0 records in
78503+0 records out
82316406784 bytes (82 GB) copied, 105.122 s, 783 MB/s
+ rm -f /EMPTY
+ exit
But I seem to have enough space:
vagrant#precise64:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/precise64-root 79G 2.3G 73G 3% /
udev 174M 0 174M 0% /dev
tmpfs 74M 272K 73M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 183M 0 183M 0% /run/shm
/dev/sda1 228M 25M 192M 12% /boot
/vagrant 220G 91G 130G 42% /vagrant
/tmp/vagrant-chef-1/chef-solo-1/cookbooks 220G 91G 130G 42% /tmp/vagrant-chef-1/chef-solo-1/cookbooks
/tmp/vagrant-chef-1/chef-solo-2/cookbooks 220G 91G 130G 42% /tmp/vagrant-chef-1/chef-solo-2/cookbooks
Can somebody please help? Thank you.
It's supposed to do this :) It's making your virtual disk as small as possible since it is thinly provisioned.
Creating a file full of zeros on the disk until it is full is clearing the blocks on the disk and as such your file representing the VMs disk will be as small as the actual data you have on the disk.
The problem resides in the following statement:
dd if=/dev/zero of=/EMPTY bs=1M
If you don't specify count=<some value>, the dd command will continue until the end of device is reached. So basically with the above command you're trying to create a file called that spawns through the whole partition, called EMPTY under /. Thus the error.