I have very little left on /, but at the same time there are more than plenty on /mnt volume, how can I use the /mnt and have all my stuff move to there?
# df -l
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 2064208 1947044 12308 100% /
/dev/sda2 153899044 192212 145889208 1% /mnt
none 873880 0 873880 0% /dev/shm
And also what's /mnt volume (/dev/sda2) for? is it EBS volume? do I got charged for using it if I move my data/binaries over?
Another solution that I am looking at is to resize the default / volume (/dev/sda2) to a bigger size, then the question would be is it possible? legitimate? and free of charge?
I wrote an article for you on how to resize the root EBS volume:
http://alestic.com/2010/02/ec2-resize-running-ebs-root
I don't recommend using /mnt ephemeral storage except for temporary, unimportant files. The content of ephemeral storage is lost when an instance is stopped or fails.
Related
I have a server on aws-ec2 with defaulit free tier. How can I increase the size of volume without using an AMI?
Here are the Steps which will help you to resize ec2 volume without AMI (Snapshots).
Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
Choose Volumes, select the volume to modify, and then choose Actions, Modify Volume.
The Modify Volume window displays the volume ID and the volume's current configuration, including type, size, and IOPS. You can change any or all of these settings in a single action. Set new configuration values as follows:
To modify the type, choose a value for Volume Type.
To modify the size, enter an allowed integer value for Size.
If you chose Provisioned IOPS (IO1) as your volume type, enter an allowed integer value for IOPS.
After you have specified all of the modifications to apply, choose Modify, Yes.
Modifying volume size has no practical effect until you also extend the volume's file system to make use of the new storage capacity.
After then you have to run these command on ec2 terminal
ubuntu#ip-192-168-1-26:~$ sudo su
root#ip-192-168-1-26:/home/ubuntu# df -h
Filesystem Size Used Avail Use% Mounted on
udev 487M 0 487M 0% /dev
tmpfs 100M 12M 88M 12% /run
/dev/xvda1 7.8G 5.5G 2.0G 74% /
tmpfs 496M 0 496M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 496M 0 496M 0% /sys/fs/cgroup
tmpfs 100M 0 100M 0% /run/user/999
tmpfs 100M 0 100M 0% /run/user/1000
root#ip-192-168-1-26:/home/ubuntu# sudo file -s /dev/xvd*
/dev/xvda: DOS/MBR boot sector
/dev/xvda1: Linux rev 1.0 ext4 filesystem data, UUID=e6d1a865-817b-456f-99e7-118135343487, volume name "cloudimg-rootfs" (needs journal recovery) (extents) (large files) (huge files)
root#ip-192-168-1-26:/home/ubuntu# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 16G 0 disk
└─xvda1 202:1 0 8G 0 part /
root#ip-192-168-1-26:/home/ubuntu# sudo growpart /dev/xvda 1
CHANGED: partition=1 start=16065 old: size=16761118 end=16777183 new: size=33538334,end=33554399
root#ip-192-168-1-26:/home/ubuntu# sudo resize2fs /dev/xvda1
resize2fs 1.42.13 (17-May-2015)
Filesystem at /dev/xvda1 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
The filesystem on /dev/xvda1 is now 4192291 (4k) blocks long.
that's command will help you to resize ec2 volume
I have created an c3.2xlarge EC2 instance with the store volume specified as 2 x 80 GB (160 GB). But when I use df -H command, this is what i see, and there is not enough storage as specified.
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.9G 62k 7.9G 1% /dev
tmpfs 7.9G 0 7.9G 0% /dev/shm
/dev/xvda1 8.4G 1.1G 7.2G 14% /
I need an EC2 instance to have at least 80 gigs of storage, which instance should I choose?
Thanks for the points in the comments.
The problem was; I used EC2 Management Console and didn't add the volumes when I created the cluster, I terminated that cluster, created a new one, on Storage page Added New Volume, chose the volume type as Instance Store 0.
[]$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdb 202:16 0 75G 0 disk /media/ephemeral0
Now the 80 Gig volume is there.
I'm using Hadoop YARN on EC2 over r3.xlarge instances, I launched the instances from an AMI using spark-ec2 scripts.
On https://aws.amazon.com/ec2/instance-types/, the specifications of r3.xlarge are the following:
vCPU: 4
Mem: 30.5 GiB
Storage: 1 x 80 GB
The Memory is good, free command gives me this result:
root#ip-xxx-xx-xx-xxx ~]$ free -g
total used free shared buffers cached
Mem: 29 2 27 0 0 1
But the storage not correspond to the indicated one.
root#ip-xxx-xx-xx-xxx ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 7.8G 6.9G 783M 91% /
devtmpfs 15G 64K 15G 1% /dev
tmpfs 15G 0 15G 0% /dev/shm
Is it normal to have only ~40GB and not 80GB like this was specified in the documentation ? Or this is because I launched the instance from an AMI ?
The two tmpfs directories aren't where your missing 80gb is. This is looks like an Debian/Ubuntu distro. I can reproduce something similar to your df:
$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 15G 12K 15G 1% /dev
tmpfs 3.0G 328K 3.0G 1% /run
/dev/xvda1 7.8G 790M 6.6G 11% /
none 15G 0 15G 0% /run/shm
Note /dev/xvda1. That's your boot partition that is on ELB. Your 80gb SSD is actually at /dev/xvdb. You need to make use of it:
mkdir -p /mnt/ssd && mkfs.ext4 /dev/xvdb \
&& echo '/dev/xvdb /mnt/ssd auto defaults,nobootwait 0 0' >> /etc/fstab \
&& mount /mnt/ssd
$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 15G 12K 15G 1% /dev
tmpfs 3.0G 328K 3.0G 1% /run
/dev/xvda1 7.8G 790M 6.6G 11% /
/dev/xvdb 74G 52M 70G 1% /mnt/ssd
Congrats! You are now the proud owner of a 80gb mount. Okay, not quite 80gb. Lets get 80gb:
$ df -H
Filesystem Size Used Avail Use% Mounted on
udev 17G 13k 17G 1% /dev
tmpfs 3.3G 336k 3.3G 1% /run
/dev/xvda1 8.4G 828M 7.1G 11% /
/dev/xvdb 80G 55M 76G 1% /mnt/ssd
Your filesystem is probably on EBS, not the instance storage that comes with r3.xlarge. This is the default for most AMIs. Note the size of the EBS volume is not part of the image. You can choose it when you create the instance.
Instance store is available on the larger instance types as shown here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html
AMI images have two options for the root storage device. The most common are EBS images, which use EBS for the root device. Since EBS isn't locked to specific hardware, these instances are much more flexible.
The other option is an AMI with an instance store root storage device. However, you lose the ability to stop the instance without terminating, change the instance type, resize the storage device, and manage the storage separately from the instance itself.
Instance store AMIs are often tagged with S3. For example: amzn-ami-hvm-2016.03.0.x86_64-s3 (ami-152bc275).
Today I started getting errors on simple operations, like creating small files in vim, the bash completion started to complain as well.
Here is the result of df -h :
vagrant#machine:/vagrant$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 40G 38G 249M 100% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 2.0G 12K 2.0G 1% /dev
tmpfs 396M 396K 395M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 2.0G 0 2.0G 0% /run/shm
none 100M 0 100M 0% /run/user
overflow 1.0M 148K 876K 15% /tmp
192.168.50.1:/Users/nha/repo/assets 233G 141G 93G 61% /var/www/assets
vagrant 233G 141G 93G 61% /vagrant
So apparently / doesn`t have space anymore ? Isn't it weird since I have space in the other filesystems (or am I misreading something) ?
How do I get more space on my vm ?
Even though you have space on your Guest OS, the VM is limited.There are couple of steps required in order to increase the size of your disk:
first, vagrant haltto close your VM
resize disk
VBoxManage clonehd box-disk1.vmdk box-disk1.vdi --format vdi
VBoxManage modifyhd box-disk1.vdi --resize 50000
start Virtual box and change configuration of the VM to associate the new disk
use fdisk to resize disk
you need to create a new partition with the new space and allocate it, so first start the VM and logged on as super user
vagrant up && vagrant ssh
su -
the command (as illustrated from my instance) are
[root#oracle ~]# fdisk /dev/sda
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): p
Disk /dev/sda: 52.4 GB, 52428800000 bytes
255 heads, 63 sectors/track, 6374 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00041a53
Device Boot Start End Blocks Id System
/dev/sda1 * 1 39 307200 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 39 2611 20663296 8e Linux LVM
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 3
First cylinder (2611-6374, default 2611):
Using default value 2611
Last cylinder, +cylinders or +size{K,M,G} (2611-6374, default 6374):
Using default value 6374
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root#oracle ~]#
note you might need to change /dev/sda compare to your configuration
create a new partition (again logged on as super user su -)
su -
[root#oracle ~]# pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 linux lvm2 a-- 19.70g 0
[root#oracle ~]# pvcreate /dev/sda3
Physical volume "/dev/sda3" successfully created
[root#oracle ~]# pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 linux lvm2 a-- 19.70g 0
/dev/sda3 lvm2 a-- 28.83g 28.83g
[root#oracle ~]# vgextend linux /dev/sda3
Volume group "linux" successfully extended
[root#oracle ~]# lvextend -l +100%FREE /dev/linux/root
[root#oracle ~]# resize2fs /dev/linux/home
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/linux/home is mounted on /home; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 2
Performing an on-line resize of /dev/linux/home to 7347200 (4k) blocks.
The filesystem on /dev/linux/home is now 7347200 blocks long.
You can increase space in your box, without losing data or creating new partitions.
Halt your VM;
Go to /home_dir/VirtualBox VMs
Change file format from .vmdk to .vdi. Then use command from the answer above to increase space.
Change the file extension back and change the file name.
Attach an extended disk to your VM.
VBoxManage storageattach <your_box_name> --storagectl "IDE Controller" --
port 0 --device 0 --type hdd --medium new_extended_file.vmdk
In your VirtualBox application go to Your_VM -> Settings -> Storage. Click on the controller and choose 'add new disk' below. Choose from existing disks the one you have just expanded.
Here's a step by step instruction how to expand the space in your vagrant box or virtual machine.
The easiest way to increase the size of the vagrant box is with the vagrant-disksize plugin.
In your vagrant root folder, run vagrant plugin install vagrant-disksize
Then add the new size to the Vagrantfile:
Vagrant.configure('2') do |config|
...
config.disksize.size = '60GB'
end
Then vagrant halt and vagrant up.
vagrant reload will not work.
I have read that the plugin has issues shrinking disk size if you overshoot.
EDIT:
On Mac, this plugin also resized the partition within the Guest OS (Ubuntu in my case).
On Windows, Vagrant reserves the space on the host OS (it enlarges the disk), but you can't use the space until resizing the partition from within the Guest OS.
I used GParted, but other solutions look simpler, such as: https://nguyenhoa93.github.io/Increase-VM-Partition
I sometimes have to destroy the machine and build it up again which in my case frees up quite a lot of space, you can do that by running
vagrant destroy
vagrant up
Please note this will result in database data being lost.
I know that the maximum slug size allowed is 200 MB. But what is the maximum disk space you can use per instance? Say I'm downloading a couple of files when the node process is running.
heroku run bash
~ $ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
- 620G 6.1G 583G 2% /tmp
You have approximately 620Gb in the /tmp folder.
Any other folder doesn't really matter, as they're readonly anyway.
When I ran df on one of my instances just now (Cedar 14), I get 304G total, 240G available. Still a lot, but less than the responder above. Just an FYI to check your specific instance.
~ $ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/evg0-evol0 304G 49G 240G 17% /tmp
I think the total size for whole server is 620GB. you can run df -k . in free account and I got 394267100 byte.. 400mb
You have approximately 620Gb in the /tmp folder. Any other folder
doesn't really matter, as they're readonly anyway.