I built a ramdisk image which is bigger than 16 MB. After downloading the ramdisk image U-boot print:
## Loading init Ramdisk from Legacy Image at 03000000 ...
Image Name: uboot ext4 ramdisk rootfs
Created: 2018-01-24 8:15:12 UTC
Image Type: AArch64 Linux RAMDisk Image (gzip compressed)
Data Size: 38319491 Bytes = 36.5 MiB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... OK
## Flattened Device Tree blob at 01000000
Booting using the fdt blob at 0x1000000
Loading Ramdisk to 3d567000, end 3f9f2583 ... OK
Using Device Tree in place at 0000000001000000, end 000000000100c554
Starting kernel ...
Timer summary in microseconds:
Mark Elapsed Stage
0 0 reset
1,106,000 1,106,000 id=64
1,437,000 331,000 id=65
1,445,000 8,000 main_loop
102,482,800,0,02,338,300,0, id=80
102,482,800,0 0 tftp_start
102,482,800,0, 0 eth_start
102,985,300,0, 5,025,000 id=81
102,985,300,0 0 id=84
102,985,300,0 0 id=82
102,985,300,0 0 tftp_done
104,389,500,0,14,042,000 bootm_start
104,734,100,0, 3,446,000 id=9
104,734,100,0 0 id=10
105,199,700,0 4,656,000 id=11
105,200,400,0 7,000 id=12
105,821,800,0 6,214,000 id=15
105,822,000,0 2,000 start_kernel
166,835,917,366,575,094,88166,835,917,355,992,894,88 board_init_f
Accumulated time:
==================================================================
Kernel is hanging.
But if the ramdisk image is smaller than 16 MB, it works.
What can I do to fix this problem?
Related
I have a socket client Golang program. when it just start up, Linux /proc/PID/status show the process RSS is 15204 kB, but the pprof report shows that HeapAlloc is about 1408 kB, there is a gap of about 14000kB.
My Questions:
1、Why is there such a big difference?
2、How is the go application memory distributed? Besides heap and stack, are there other memory areas? and how can I find these areas?
3、More importantly, how can I lower its rss?
cat /proc/PID/status:
Umask: 0000
State: S (sleeping)
Tgid: 3393
Ngid: 0
Pid: 3393
PPid: 2882
TracerPid: 0
Uid: 500 500 500 500
Gid: 500 500 500 500
FDSize: 32
Groups: 500
NStgid: 3393
NSpid: 3393
NSpgid: 2881
NSsid: 2881
VmPeak: 806492 kB
VmSize: 806492 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 15204 kB
VmRSS: 15204 kB
RssAnon: 5024 kB
RssFile: 10180 kB
RssShmem: 0 kB
VmData: 10988 kB
VmStk: 132 kB
VmExe: 5164 kB
VmLib: 8 kB
VmPTE: 28 kB
VmPMD: 0 kB
VmSwap: 0 kB
Threads: 6
SigQ: 0/937
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000000001
SigCgt: fffffffe7fc1fefe
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000
Cpus_allowed: 3
Cpus_allowed_list: 0-1
voluntary_ctxt_switches: 261951
nonvoluntary_ctxt_switches: 21327
go tool pprof heap:
# runtime.MemStats
# Alloc = 1408048
# TotalAlloc = 45071968
# Sys = 10828924
# Lookups = 0
# Mallocs = 889174
# Frees = 885421
# HeapAlloc = 1408048
# HeapSys = 7929856
# HeapIdle = 5677056
# HeapInuse = 2252800
# HeapReleased = 5480448
# HeapObjects = 3753
# Stack = 458752 / 458752
# MSpan = 25120 / 32768
# MCache = 1736 / 16384
# BuckHashSys = 725549
# GCSys = 886912
# OtherSys = 778703
# NextGC = 4194304
# LastGC = 1645757614280889245
"Why is there such a big difference?"
These are two different metrics. What people call "memory consumption" is
incredibly hard to even define on modern machines.
"How is the go application memory distributed?"
This is an implementation detail and varies. Nothing actuable to know here.
"Besides heap and stack, are there other memory areas?"
No, but note that the heap/stack dichotomy is an implementation detail of a certain (albeit common) compiler/compiler version/system combination.
"and how can I find these areas?" You cannot.
"More importantly, how can I lower its rss?"
By reducing how much memory you allocate. But note that lowering RSS most probably simply isn't needed. You probably overestimate how "problematic" a "large" RSS is.
I want to run linux 3.10 with mips64r2 on qemu. But it fails, the boot log as follows,
I compile the kernel with the gcc 4.9.3 which is modified by loongson.
The kernel config file is the malta_defconfig and i change it to mips64r2 cpu and 64 bit kernel.
The qemu 2.5 is the default application on the ubuntu 16.04.
zlp#lab302i-ES:~/projs/linux-3.10$ qemu-system-mips64el -M malta -m 1G -cpu 5KEf -kernel vmlinux -nographic
Linux version 3.10.0 (zlp#lab302i-ES) (gcc version 4.9.3 20150626 (Red Hat 4.9.3-2) (GCC) ) #8 SMP Tue Nov 19 19:16:32 CST 2019
Config serial console: console=ttyS0,38400n8r
bootconsole [early0] enabled
CPU revision is: 00018900 (MIPS 5KE)
FPU revision is: 00738900
Checking for the multiply/shift bug... no.
Checking for the daddiu bug... no.
Software DMA cache coherency enabled
Determined physical RAM map:
memory: 0000000000001000 # 0000000000000000 (reserved)
memory: 00000000000ef000 # 0000000000001000 (ROM data)
memory: 0000000000539000 # 00000000000f0000 (reserved)
memory: 000000000f9d7000 # 0000000000629000 (usable)
Wasting 88312 bytes for tracking 1577 unused pages
Zone ranges:
DMA [mem 0x00000000-0x00ffffff]
Normal [mem 0x01000000-0x0fffffff]
Movable zone start for each node
Early memory node ranges
node 0: [mem 0x00000000-0x0fffffff]
Primary instruction cache 8kB, VIPT, 2-way, linesize 32 bytes.
Primary data cache 8kB, 2-way, VIPT, no aliases, linesize 32 bytes
PERCPU: Embedded 10 pages/cpu #9800000001384000 s10816 r8192 d21952 u40960
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 64640
Kernel command line: console=ttyS0,38400n8r
PID hash table entries: 1024 (order: 1, 8192 bytes)
Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
Inode-cache hash table entries: 16384 (order: 5, 131072 bytes)
Cache parity protection disabled
Memory: 251796k/255836k available (3695k kernel code, 4040k reserved, 1150k data, 272k init, 0k highmem)
Hierarchical RCU implementation.
CONFIG_RCU_FANOUT set to non-default value of 32
RCU restricting CPUs from NR_CPUS=2 to nr_cpu_ids=1.
NR_IRQS:256
CPU frequency 200.00 MHz
Console: colour dummy device 80x25
Calibrating delay loop... 1076.42 BogoMIPS (lpj=5382144)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 256
Checking for the daddi bug... no.
Brought up 1 CPUs
NET: Registered protocol family 16
bio: create slab <bio-0> at 0
vgaarb: loaded
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x10000000-0x17ffffff]
pci_bus 0000:00: root bus resource [io 0x2000-0x1fffff]
pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
pci 0000:00:0a.3: no compatible bridge window for [io 0x1100-0x110f]
vgaarb: device added: PCI:0000:00:12.0,decodes=io+mem,owns=none,locks=none
pci 0000:00:0a.3: BAR 8: [io 0x1100-0x110f] has bogus alignment
pci 0000:00:12.0: BAR 0: assigned [mem 0x10000000-0x11ffffff pref]
pci 0000:00:0b.0: BAR 6: assigned [mem 0x12000000-0x1203ffff pref]
pci 0000:00:12.0: BAR 6: assigned [mem 0x12040000-0x1204ffff pref]
pci 0000:00:12.0: BAR 1: assigned [mem 0x12050000-0x12050fff]
pci 0000:00:0a.2: BAR 4: assigned [io 0x2000-0x201f]
pci 0000:00:0b.0: BAR 0: assigned [io 0x2020-0x203f]
pci 0000:00:0b.0: BAR 1: assigned [mem 0x12051000-0x1205101f]
pci 0000:00:0a.1: BAR 4: assigned [io 0x2040-0x204f]
Switching to clocksource pit
NET: Registered protocol family 2
TCP established hash table entries: 2048 (order: 3, 32768 bytes)
TCP bind hash table entries: 2048 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
TCP: reno registered
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
PCI: Enabling device 0000:00:0a.2 (0000 -> 0001)
CPU 0 Unable to handle kernel paging request at virtual address 0000000000000000, epc == ffffffff8016bef4, ra == ffffffff805c51a0
Oops[#1]:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.10.0 #8
task: 980000000fc433f0 ti: 980000000fc44000 task.ti: 980000000fc44000
$ 0 : 0000000000000000 0000000000000008 0000000000000000 0000000000000000
$ 4 : 980000000fc47de0 0000000000000000 0000000000000000 0000000000000000
$ 8 : 0000000000000000 980000000fd84e60 fffffffffffffffc ffffffff8061cd30
$12 : 0000000000000010 ffffffff802e3bd4 0000000000000000 000000000000001a
$16 : ffffffff80600000 ffffffff805c5184 0000000000000000 ffffffff80600000
$20 : ffffffff805e6368 ffffffff805e6338 ffffffff805bc1d8 ffffffff805e62f8
$24 : 0000000000000018 ffffffff803451b0
$28 : 980000000fc44000 980000000fc47de0 ffffffff80600000 ffffffff805c51a0
Hi : 0000000000000001
Lo : 1111111111111112
epc : ffffffff8016bef4 cmpxchg_futex_value_locked+0x2c/0x78
Not tainted
ra : ffffffff805c51a0 futex_init+0x1c/0x6c
Status: 1400a7e3 KX SX UX KERNEL EXL IE
Cause : 00800008
BadVA : 0000000000000000
PrId : 00018900 (MIPS 5KE)
Modules linked in:
Process swapper/0 (pid: 1, threadinfo=980000000fc44000, task=980000000fc433f0, tls=0000000000000000)
Stack : ffffffff805c4f3c 0000000000000000 ffffffff80600000 ffffffff801004f0
ffffffff805e6368 0000000000000006 0000000000000030 ffffffff805f0a30
ffffffff80600000 ffffffff805bca24 0000000000000066 0000000000000000
ffffffff80494a48 0000000000000000 ffffffff80600000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 ffffffff80494a64 ffffffff80494a48 0000000000000000
0000000000000000 ffffffff80101f18 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
...
Call Trace:
[<ffffffff8016bef4>] cmpxchg_futex_value_locked+0x2c/0x78
[<ffffffff805c51a0>] futex_init+0x1c/0x6c
[<ffffffff801004f0>] do_one_initcall+0xe0/0x160
[<ffffffff805bca24>] kernel_init_freeable+0x16c/0x220
[<ffffffff80494a64>] kernel_init+0x1c/0x160
[<ffffffff80101f18>] ret_from_kernel_thread+0x14/0x1c
Code: 00000000 0000102d 0000000f <c0a30000> 14660005 00000000 00e0082d e0a10000 1020fff9
---[ end trace 47a33b7db369802c ]---
note: swapper/0[1] exited with preempt_count 1
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
Several possible issues here:
(1) QEMU 2.5 is now pretty old. You could retry with something more modern.
(2) You're building the kernel with "a gcc modified by loongson", but you're not actually running this on a Loongson CPU. Try using a stock gcc to build your kernel, and make sure that your kernel config matches the board model and CPU model you're asking QEMU to emulate. (The kernel log you give shows it crashing in a low level function which is going to try to do an atomic cmpxchg operation, and I have a vague recollection that this is an area where different MIPS CPUs have different sets of supported instructions, and in particular that Loongson might have made some changes here. So my first guess would be that your problem is here.)
I have an embedded system with power pc architecture 87xx. If I don't enable the "CONFIG_CGROUPS" in Linux kernel I can't mount the devtmpf i.e. ubifs file system.I am trying to understand what't the dependency of CGROUPS in mounting the nand filesystem running UBIFS.If I enable the CGROUPS then I can't execute the /sbin/init of the mounted root file system. What I could be doing wrong here?
This happens if I don't turned the CONFIG_CGROUPS=y in kernel.
Zone ranges:
DMA [mem 0x0000000000000000-0x000000001fffffff]
Normal empty
Movable zone start for each node
Early memory node ranges
node 0: [mem 0x0000000000000000-0x000000001fffffff]
Initmem setup node 0 [mem 0x0000000000000000-0x000000001fffffff]
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048
Kernel command line: ubi.mtd=1 rootfstype=ubifs root=ubi0:discofs rw console=tty S1,115200 serno=91101316 sku=700064090C hwrev==00:90:5E:12:32:05 mac1=00:90:5E:1 2:32:06 mac2=00:90:5E:12:32:07 mac3=00:90:5E:12:32:08 mac4=00:90:5E:12:32:09
PID hash table entries: 2048 (order: 1, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 511944K/524288K available (5972K kernel code, 272K rwdata, 1192K rodata, 192K init, 128K bss, 12344K reserved, 0K cma-reserved)
Kernel virtual memory layout:
* 0xfffdf000..0xfffff000 : fixmap
* 0xfdffe000..0xfe000000 : early ioremap
* 0xe1000000..0xfdffe000 : vmalloc & ioremap
NR_IRQS:512 nr_irqs:512 16
IPIC (128 IRQ sources) at e1000700
clocksource: timebase: mask: 0xffffffffffffffff max_cycles: 0x12049cd416, max_id le_ns: 440795202745 ns
clocksource: timebase mult[ccccccd] shift[24] registered
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645 041785100000 ns
futex hash table entries: 256 (order: -1, 3072 bytes)
NET: Registered protocol family 16
PCI: Probing PCI hardware
Freescale Elo series DMA driver
vgaarb: loaded
SCSI subsystem initialized
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti#l inux.it>
PTP clock support registered
clocksource: Switched to clocksource timebase
NET: Registered protocol family 2
TCP established hash table entries: 4096 (order: 2, 16384 bytes)
TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 256 (order: 0, 4096 bytes)
UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
fsl-elo-dma e00082a8.dma: #0 (fsl,elo-dma-channel), irq 71
fsl-elo-dma e00082a8.dma: #1 (fsl,elo-dma-channel), irq 71
fsl-elo-dma e00082a8.dma: #2 (fsl,elo-dma-channel), irq 71
fsl-elo-dma e00082a8.dma: #3 (fsl,elo-dma-channel), irq 71
workingset: timestamp_bits=30 max_order=17 bucket_order=0
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
SGI XFS with security attributes, no debug enabled
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Oops: Exception in kernel mode, sig: 4 [#1]
disco
Modules linked in:
CPU: 0 PID: 8 Comm: kdevtmpfs Not tainted 4.12.28-disco-standard #1
task: df41b580 task.stack: df438000
NIP: c0100000 LR: c00fb708 CTR: c00b462c
REGS: df439c50 TRAP: 0700 Not tainted (4.12.28-disco-standard)
MSR: 00089032 <EE,ME,IR,DR,RI>
CR: 880ad872 XER: 00000000
GPR00: c00fb708 df439d00 df41b580 df69c170 00000000 df69c170 00000002 00000000
GPR08: 00000000 00000007 00000001 0e4e1c2f 280ad872 00000000 c0040514 df41d120
GPR16: 00000000 014000c0 00000007 00000000 00000000 0000002f 00000000 c078c840
GPR24: fffff000 00000000 00000000 df69c1cc df69c170 df41fd18 00000000 df69c170
NIP [c0100000] iput+0x24/0x1c8
LR [c00fb708] d_delete+0xb8/0xfc
Call Trace:
[df439d00] [c00fb000] dentry_unlink_inode+0xf0/0x164 (unreliable)
[df439d20] [c00fb708] d_delete+0xb8/0xfc
[df439d40] [c00eecc4] vfs_unlink+0x1b8/0x210
[df439d70] [c03c4f2c] handle_remove+0x1b0/0x324
[df439e50] [c03c5200] devtmpfsd+0x160/0x34c
[df439f00] [c0040650] kthread+0x13c/0x140
[df439f40] [c00103f0] ret_from_kernel_thread+0x5c/0x64
Instruction dump:
39200001 7d234b78 4e800020 9421ffe0 93e1001c 7c7f1b79 41820094 7c0802a6
93a10014 90010024 93c10018 813f0058 <27051956> 9188a4bd 5d2e9c29 0039c48b
---[ end trace df78166105247941 ]---
My guess is you have a systemd-based system. In which case, init is provided by systemd, so is population of /dev. Systemd, in turn, makes extensive use of CGROUPS.
The ubifs/devtmpfs link is not a direct one, IMO. ubifs could be used without it too as long as /dev is correctly populated.
So, I'm trying to run some simple code, jdk-8, output via jol
System.out.println(VMSupport.vmDetails());
Integer i = new Integer(23);
System.out.println(ClassLayout.parseInstance(i)
.toPrintable());
The first attempt is to run it with compressed oops disabled and compressed klass also on 64-bit JVM.
-XX:-UseCompressedOops -XX:-UseCompressedClassPointers
The output, pretty much expected is :
Running 64-bit HotSpot VM.
Objects are 8 bytes aligned.
java.lang.Integer object internals:
OFFSET SIZE TYPE DESCRIPTION VALUE
0 4 (object header) 01 00 00 00 (00000001 00000000 00000000 00000000) (1)
4 4 (object header) 00 00 00 00 (00000000 00000000 00000000 00000000) (0)
8 4 (object header) 48 33 36 97 (01001000 00110011 00110110 10010111) (-1758055608)
12 4 (object header) 01 00 00 00 (00000001 00000000 00000000 00000000) (1)
16 4 int Integer.value 23
20 4 (loss due to the next object alignment)
Instance size: 24 bytes (reported by Instrumentation API)
Space losses: 0 bytes internal + 4 bytes external = 4 bytes total
That makes sense : 8 bytes klass word + 8 bytes mark word + 4 bytes for the actual value and 4 for padding (to align on 8 bytes) = 24 bytes.
The second attempt it to run it with compressed oops enabled compressed klass also on 64-bit JVM.
Again, the output is pretty much understandable :
Running 64-bit HotSpot VM.
Using compressed oop with 3-bit shift.
Using compressed klass with 3-bit shift.
Objects are 8 bytes aligned.
OFFSET SIZE TYPE DESCRIPTION VALUE
0 4 (object header) 01 00 00 00 (00000001 00000000 00000000 00000000) (1)
4 4 (object header) 00 00 00 00 (00000000 00000000 00000000 00000000) (0)
8 4 (object header) f9 33 01 f8 (11111001 00110011 00000001 11111000) (-134138887)
12 4 int Dummy.i 42
Instance size: 16 bytes (reported by Instrumentation API).
4 bytes compressed oop (klass word) + 8 bytes mark word + 4 bytes for the value + no space loss = 16 bytes.
The thing that does NOT make sense to me is this use-case:
-XX:+UseCompressedOops -XX:+UseCompressedClassPointers -XX:ObjectAlignmentInBytes=16
The output is this:
Running 64-bit HotSpot VM.
Using compressed oop with 4-bit shift.
Using compressed klass with 0x0000001000000000 base address and 0-bit shift.
I was really expecting to both be "4-bit shift". Why they are not?
EDIT
The second example is run with :
XX:+UseCompressedOops -XX:+UseCompressedClassPointers
And the third one with :
-XX:+UseCompressedOops -XX:+UseCompressedClassPointers -XX:ObjectAlignmentInBytes=16
Answers to these questions are mostly easy to figure out when looking into OpenJDK code.
For example, grep for "UseCompressedClassPointers", this will get you to arguments.cpp:
// Check the CompressedClassSpaceSize to make sure we use compressed klass ptrs.
if (UseCompressedClassPointers) {
if (CompressedClassSpaceSize > KlassEncodingMetaspaceMax) {
warning("CompressedClassSpaceSize is too large for UseCompressedClassPointers");
FLAG_SET_DEFAULT(UseCompressedClassPointers, false);
}
}
Okay, interesting, there is "CompressedClassSpaceSize"? Grep for its definition, it's in globals.hpp:
product(size_t, CompressedClassSpaceSize, 1*G, \
"Maximum size of class area in Metaspace when compressed " \
"class pointers are used") \
range(1*M, 3*G) \
Aha, so the class area is in Metaspace, and it takes somewhere between 1 Mb and 3 Gb of space. Let's grep for "CompressedClassSpaceSize" usages, because that will take us to actual code that handles it, say in metaspace.cpp:
// For UseCompressedClassPointers the class space is reserved above
// the top of the Java heap. The argument passed in is at the base of
// the compressed space.
void Metaspace::initialize_class_space(ReservedSpace rs) {
So, compressed classes are allocated in a smaller class space outside the Java heap, which does not require shifting -- even 3 gigabytes is small enough to use only the lowest 32 bits.
I will try to extend a little bit on the answer provided by Alexey as some things might not be obvious.
Following Alexey suggestion, if we search the source code of OpenJDK for where compressed klass bit shift value is assigned, we will find the following code in metaspace.cpp:
void Metaspace::set_narrow_klass_base_and_shift(address metaspace_base, address cds_base) {
// some code removed
if ((uint64_t)(higher_address - lower_base) <= UnscaledClassSpaceMax) {
Universe::set_narrow_klass_shift(0);
} else {
assert(!UseSharedSpaces, "Cannot shift with UseSharedSpaces");
Universe::set_narrow_klass_shift(LogKlassAlignmentInBytes);
}
As we can see, the class shift can either be 0(or basically no shifting) or 3 bits, because LogKlassAlignmentInBytes is a constant defined in globalDefinitions.hpp:
const int LogKlassAlignmentInBytes = 3;
So, the answer to your quetion:
I was really expecting to both be "4-bit shift". Why they are not?
is that ObjectAlignmentInBytes does not have any effect on compressed class pointers alignment in the metaspace which is always 8bytes.
Of course this conclusion does not answer the question:
"Why when using -XX:ObjectAlignmentInBytes=16 with -XX:+UseCompressedClassPointers the narrow klass shift becomes zero? Also, without shifting how can the JVM reference the class space with 32-bit references, if the heap is 4GBytes or more?"
We already know that the class space is allocated on top of the java heap and can be up to 3G in size. With that in mind let's make a few tests. -XX:+UseCompressedOops -XX:+UseCompressedClassPointers are enabled by default, so we can eliminate these for conciseness.
Test 1: Defaults - 8 Bytes aligned
$ java -XX:ObjectAlignmentInBytes=8 -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode -version
heap address: 0x00000006c0000000, size: 4096 MB, zero based Compressed Oops
Narrow klass base: 0x0000000000000000, Narrow klass shift: 3
Compressed class space size: 1073741824 Address: 0x00000007c0000000 Req Addr: 0x00000007c0000000
Notice that the heap starts at address 0x00000006c0000000 in the virtual space and has a size of 4GBytes. Let's jump by 4Gbytes from where the heap starts and we land just where class space begins.
0x00000006c0000000 + 0x0000000100000000 = 0x00000007c0000000
The class space size is 1Gbyte, so let's jump by another 1Gbyte:
0x00000007c0000000 + 0x0000000040000000 = 0x0000000800000000
and we land just below 32Gbytes. With a 3 bits class space shifting the JVM is able to reference the entire class space, although it's at the limit (intentionally).
Test 2: 16 bytes aligned
java -XX:ObjectAlignmentInBytes=16 -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode -version
heap address: 0x0000000f00000000, size: 4096 MB, zero based Compressed Oops
Narrow klass base: 0x0000001000000000, Narrow klass shift: 0
Compressed class space size: 1073741824 Address: 0x0000001000000000 Req Addr: 0x0000001000000000
This time we can observe that the heap address is different, but let's try the same steps:
0x0000000f00000000 + 0x0000000100000000 = 0x0000001000000000
This time around heap space ends just below 64GBytes virtual space boundary and the class space is allocated above 64Gbyte boundary. Since class space can use only 3 bits shifting, how can the JVM reference the class space located above 64Gbyte? The key is:
Narrow klass base: 0x0000001000000000
The JVM still uses 32 bit compressed pointers for the class space, but when encoding and decoding these, it will always add 0x0000001000000000 base to the compressed reference instead of using shifting. Note, that this approach works as long as the referenced chunk of memory is lower than 4Gbytes (the limit for 32 bits references). Considering that the class space can have a maximum of 3Gbytes we are comfortably within the limits.
3: 16 bytes aligned, pin heap base at 8g
$ java -XX:ObjectAlignmentInBytes=16 -XX:HeapBaseMinAddress=8g -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode -version
heap address: 0x0000000200000000, size: 4096 MB, zero based Compressed Oops
Narrow klass base: 0x0000000000000000, Narrow klass shift: 3
Compressed class space size: 1073741824 Address: 0x0000000300000000 Req Addr: 0x0000000300000000
In this test we are still keeping the -XX:ObjectAlignmentInBytes=16, but also asking the JVM to allocate the heap at the 8th GByte in the virtual address space using -XX:HeapBaseMinAddress=8g JVM argument. The class space will begin at 12th GByte in the virtual address space and 3 bits shifting is more than enough to reference it.
Hopefully, these tests and their results answer the question:
"Why when using -XX:ObjectAlignmentInBytes=16 with -XX:+UseCompressedClassPointers the narrow klass shift becomes zero? Also, without shifting how can the JVM reference the class space with 32-bit references, if the heap is 4GBytes or more?"
I have cPanel/WHM installed on a 40gb partition, however WHM shows that 8.9gb out of 9.9gb is in use. How do I correct this?
This is on an AWS EC2 instance. The root volume is configured to 40gb.
After running df -h :
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
9.9G 8.9G 574M 95% /
/dev/hda1 99M 26M 69M 28% /boot
tmpfs 1006M 0 1006M 0% /dev/shm
So that shows that the /dev/mapper/VolGroup00-LogVol00 is 9.9GB. However, if I run parted and print the configuration I can see that:
Model: QEMU HARDDISK (ide)
Disk /dev/hda: 42.9GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 32.3kB 107MB 107MB primary ext3 boot
2 107MB 21.5GB 21.4GB primary lvm
I need the whole 40GB for cPanel/WHM. Why would it limit its self to 1/4 of the disk?
After Running vgs
VG #PV #LV #SN Attr VSize VFree
VolGroup00 1 2 0 wz--n- 19.88G 0
pvs:
PV VG Fmt Attr PSize PFree
/dev/hda2 VolGroup00 lvm2 a-- 19.88G 0
lvs:
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
LogVol00 VolGroup00 -wi-ao 10.22G
LogVol01 VolGroup00 -wi-ao 9.66G
fdisk -l
Disk /dev/hda: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 13 104391 83 Linux
/dev/hda2 14 2610 20860402+ 8e Linux LVM
Disk /dev/dm-0: 10.9 GB, 10972299264 bytes
255 heads, 63 sectors/track, 1333 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/dm-0 doesn't contain a valid partition table
Disk /dev/dm-1: 10.3 GB, 10368319488 bytes
255 heads, 63 sectors/track, 1260 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/dm-1 doesn't contain a valid partition table
Where are you checking disk usages in WHM ? Can you please let me know following command out put so that I can assist you on this.
df -h
I think there is free space on your server in LVM partition. Can you please check this with the following command and let me know
vgs
pvs
lvs
fdisk -l
And if you found any free space in your VolGroup, Then you will have to increase it through lvextend command, You can check it at http://www.24x7servermanagement.com/blog/how-to-increase-the-size-of-the-logical-volume/