Update:
So, restarting the mac did the work but any known reason for this bug?
I'm using vm_stat to calculate RAM info like here
But when I try to add all the values and multiply it with page size im getting approx. 1.3gb less.
Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free: 22064.
Pages active: 580105.
Pages inactive: 472217.
Pages speculative: 5594.
Pages throttled: 0.
Pages wired down: 559999.
Pages purgeable: 29101.
"Translation faults": 261945239.
Pages copy-on-write: 6941679.
Pages zero filled: 165324784.
Pages reactivated: 14573079.
Pages purged: 1602247.
File-backed pages: 203023.
Anonymous pages: 854893.
Pages stored in compressor: 1732046.
Pages occupied by compressor: 456427.
Decompressions: 11423912.
Compressions: 20641865.
Pageins: 4475678.
Pageouts: 32877.
Swapins: 1714616.
Swapouts: 2389086.
So by adding first 6 values and multiplying with page size, im getting 6.7GB but my mac has 8GB.
So what is going wrong?
Thank you!
Here is the result
After the reboot
It looks to me like you need to add in "Pages occupied by compressor". That gets you to almost exactly 8GB. The reboot probably just reset that to zero so it didn't matter.
Related
Why are entries in the Page Global Directory offset? What is the significance of the offset, if any?
Page Global Directory
Address
Entry 1
Entry 2
0000000080036000:
0x0000000000000000
0x0000000000000000
...
...
...
0000000080036bf0:
0x0000000000000000
0x0000000000000000
0000000080036c00:
0x000000002000d401
0x0000000000000000
0000000080036c10:
0x0000000000000000
0x0000000000000000
...
...
...
0000000080036ff0:
0x0000000000000000
0x0000000000000000
Why not start the entries at position 0 at 0x80036000?
Linux and seL4 for RISC-V (sv39) use a three level page table that has a Page Global Directory (PGD), Page Middle Directory (PMD), and Page Table Entries (PTE). The PTE entries point to executable data. It's exactly this: http://www.science.unitn.it/~fiorella/guidelinux/tlk/node34.html
Each table is 4096 bytes or 0x1000. The entries are 64 bytes each. Each table can have 512 entries (0x200). Some of the tables--particularly PGD and PMD have entries that are offset. In other words, instead of starting at position 0 in the table, the entries are 0 until halfway or even 3/4 of the way through the table. I'm trying to understand why that is.
The question is about the location within the table rather than the contents of the location. That is why start at 0x80036c00 vs. what 0x2000d401 means? I know that 0x2000d401 points to an entry in the PMD which points to an entry in PTE and that finally points to executable code.
I can walk the page table conceptually without a problem. My problem is that I've moved the payload in my binary and modified the page tables to use 4KB instead of 2MB pages. This works in a special case, but no the general case and I'm trying to understand why. I suspect I've got something wrong in the page tables based on QEMU outputs I get.
Why are entries in the Page Global Directory offset? What is the significance of the offset, if any?
The entries in the PGD are not offsets they are pointers to another page table. The offsets in the page tables are contained within the virtual address.
Linux and seL4 use a three level page table that has a Page Global Directory (PGD), Page Middle Directory (PMD), and Page Table Entries (PTE).
To my knowledge, this is outdated information. I don't know anything about seL4 but Linux uses 4 level page tables. This probably also depends on the architecture like x86-64 vs ARM. Even then, I'm quite sure Linux uses 4 levels regardless of the architecture and simply discards some levels based on the architecture's requirement.
Each table is 4096 bytes or 0x1000. The entries are 64 bytes each. Each table can have 512 entries (0x200). Some of the tables--particularly PGD and PMD have entries that are offset. In other words, instead of starting at position 0 in the table, the entries are 0 until halfway or even 3/4 of the way through the table. I'm trying to understand why that is.
This paragraph gives me a hint (maybe I'm wrong) that your question is regarding the x86-64 architecture. On x86-64 there is 4 levels of page tables. Paging is partially implemented by hardware in that the MMU of the processor will automatically cross the page tables to translate virtual addresses to physical ones. It is also partially implemented by software (the operating-system) in that the OS will fill the page tables and such.
There are several reasons why some portions of the page tables could be zero. What I'm thinking of is maybe the portion of the memory represented by the page table entry is not in the address space of the process. This means that a user mode process can access this address but it will trigger a page fault and the kernel should thus kill the process.
Also, if it is a kernel page table, it could only be that a portion of the virtual address space was currently unused by the kernel and can thus be discarded by placing zeroes in the different entries. Several reasons can explain why some portions of the page tables are zero.
It is certain that it is always a good idea to fill the page tables with zeroes before "giving" the page table to a user mode process because otherwise an access that touches the address space of another process could happen and this would be a security threat. For example, if you have an entry in the page tables of a user mode process that wasn't zeroed out, you could have a process which accesses that virtual address. It could have the present bit set (it would not trigger a page fault) and could translate to the address space of another process. This would be a vulnerability.
Don't forget that page tables are also memory protection because they isolate one process from another in that the page tables should not translate to addresses belonging to another process. They also have the user vs supervisor bit which allows to isolate user mode from the kernel.
As to paging, the 4 level paging scheme on x86-64 has PML4, PGD, PDT and PT. The virtual address looks like the following (in binary):
Index in PML4 Index in PGD Index in PDT Index in PT Offset in physical frame (12 bits)
0b000000000 000000000 000000000 000000000 000000000000
Here I represented only 48 bits because the upper 16 bits are unused (unless 5 level paging is enabled which is present on newer processors). Basically, each 9 bits is an index/offset in the corresponding page table. The 12 least significant bits are the offset in the physical frame. Linux simply gave general names to its page tables because it supports several architectures. The names on Linux are PGD, PUD (Page Upper Directory), PMD and PTE.
The page table entry points to another page table which will be used as the table for the translation using the offset found in the virtual address.
Hope this clears any misconceptions! Feel free to comment for anything.
I have change my unsecure base url from {{base_url}} to {{unsecure_base_url}}.
After this I am getting this error
"Fatal error:Allowed memory size of 268435456 bytes exhausted (tried to allocate 65488 bytes)".
I have tried ti change set ini_set('memory_limit', '512M').
Now it show the error "Fatal error:Allowed memory size of 536870912 bytes exhausted (tried to allocate 65488 bytes)".
I also changed the values in databse core_config_data > unsecure/base_url” andchange the value to “{{base_url}}”;but still have the same error. Can anyone help me?.
Thanks in advance.
It sounds like your changes will do something after clearing cache. You can delete files in
{Magento dir}/var/cache
The fact that setting the limit to 512M does not help, is that it needs even more (sounds like a leak to me). Usually something is wrong, but you could try to increase memory usage until you reach a sane number.
The official documentation states a minimum of 254 and a required 512 megabyte of ram. Maybe the usage may increase when modules/plugins are added, custom layouts are used, etc.
http://magento.com/resources/previous-magento-system-requirements
I would like to know if there is a command/API call (or set of commands/API calls) that calculates each of (Virtual Memory, File Cache and App Memory) parameters listed in the screen shot above.
You can use vm_stat and sysctl terminal commands. Although there was no straightforward way or documentation on how to extract the new attributes from these commands, we had to do some trial and error till we discovered the relations between parameters in the commands and the attribute we need to calculate.
The Steps are as the following:
Run vm_stat
Run "sysctl hw.memsize" and "sysctl vm.swapusage".
The relationship between Memory usage which appears in Activity Monitor and previous commands are described in How to calc Memory usage in Mavericks programmatically.
Sample output from vm_stat:
Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free: 24428.
Pages active: 1039653.
Pages inactive: 626002.
Pages speculative: 184530.
Pages throttled: 0.
Pages wired down: 156244.
Pages purgeable: 9429.
"Translation faults": 14335334.
Pages copy-on-write: 557301.
Pages zero filled: 5682527.
Pages reactivated: 74.
Pages purged: 52633.
File-backed pages: 660167.
Anonymous pages: 1190018.
Pages stored in compressor: 644.
Pages occupied by compressor: 603.
Decompressions: 18.
Compressions: 859.
Pageins: 253589.
Pageouts: 0.
Swapins: 0.
Swapouts: 0.
reading the copy on write about window's memory management, it is saying that system will find a free page in the RAM for the shared memory ( be backed immediately by disk page ).
why it is necessary to back the RAM page with disk page ? it is not swapped out, it is just created ?
I remember the RAM page only get swapped when there is not enough RAM page
The system needs the guarantee that when a write happens, space will be available. You can't fail an allocation now if the system will run out of diskspace later.
That doesn't mean the disk s written to; the page reservation is merely bookkeeping.
In Vista Task Manager, I understand the available page file is listed like this:
Page File inUse M / available M
In XP it's listed as the Commit Charge Limit.
I had thought that:
Available Virtual Memory = Physical Memory Total + Sum of Page Files
But on my machine I've got Physical Memory = 2038M, Page Files = 4096M, Page File Available = 6051. There's 83M unaccounted for here. What's that used for. I thought it might be something to do with the Kernel memory, but the number doesn't seem to match up?
Info I've found so far:
See http://msdn.microsoft.com/en-us/library/aa965225(VS.85).aspx for more info.
Page file size can be found here: Computer Properties, advanced, performance settings, advanced.
I think you are correct in your guess it has to do something with the kernel - the kernel memory needs some physical backup as well.
However I have to admit that when trying to verify try, the numbers still do not match well and there is a significant amount of memory not accounted for by this.
I have:
Available Virtual Memory = 4 033 552 KB
Physical Memory Total = 2 096 148 KB
Sum of Page Files = 2048 MB
Kernel Non-Paged Memory = 28 264 KB
Kernel Paged Memory = 63 668 KB