Understand kernel panic / advice - linux-kernel

I'm using debian jessie with kernel 3.16.39-1:
# apt-cache policy linux-image-3.16.0-4-amd64
linux-image-3.16.0-4-amd64:
Installed: 3.16.39-1
Candidate: 3.16.39-1
Version table:
*** 3.16.39-1 0
500 http://ftp.fr.debian.org/debian/ jessie/main amd64 Packages
100 /var/lib/dpkg/status
This machine is using 2 bonding interface:
bond0: 2*10Gb/s ixgbe X520
bond1: 2*10Gb/s ixgbe X520
irqbalance is running on this machine.
Under network load (12Gb/s on bond1) i got the following kernel panic:
kernel: [26339.017497] Call Trace:
kernel: [26339.017499] <IRQ> [<ffffffff81514c11>] ? dump_stack+0x5d/0x78
kernel: [26339.017509] [<ffffffff81144a3f>] ? warn_alloc_failed+0xdf/0x130
kernel: [26339.017513] [<ffffffff810a949d>] ? __wake_up_sync_key+0x3d/0x60
kernel: [26339.017515] [<ffffffff81148daf>] ? __alloc_pages_nodemask+0x8ef/0xb50
kernel: [26339.017519] [<ffffffff8147eaff>] ? tcp_v4_do_rcv+0x1af/0x4c0
kernel: [26339.017524] [<ffffffff81455b66>] ? nf_hook_slow+0x76/0x130
kernel: [26339.017528] [<ffffffff811883ad>] ? alloc_pages_current+0x9d/0x150
kernel: [26339.017531] [<ffffffff81412d7b>] ? __netdev_alloc_frag+0x8b/0x140
kernel: [26339.017534] [<ffffffff8141913f>] ? __netdev_alloc_skb+0x6f/0xf0
kernel: [26339.017558] [<ffffffffa0146a0d>] ? ixgbe_clean_rx_irq+0x10d/0xb70 [ixgbe]
kernel: [26339.017564] [<ffffffffa0148198>] ? ixgbe_poll+0x488/0x860 [ixgbe]
kernel: [26339.017567] [<ffffffff8108c9ad>] ? hrtimer_get_next_event+0xad/0xc0
kernel: [26339.017570] [<ffffffff81425509>] ? net_rx_action+0x129/0x250
kernel: [26339.017573] [<ffffffff8106d911>] ? __do_softirq+0xf1/0x2d0
kernel: [26339.017575] [<ffffffff8106dd25>] ? irq_exit+0x95/0xa0
kernel: [26339.017578] [<ffffffff8151dbe2>] ? do_IRQ+0x52/0xe0
kernel: [26339.017582] [<ffffffff8151ba2d>] ? common_interrupt+0x6d/0x6d
kernel: [26339.017583] <EOI> [<ffffffff8108c31d>] ? __hrtimer_start_range_ns+0x1cd/0x3a0
kernel: [26339.017588] [<ffffffff813e32a2>] ? cpuidle_enter_state+0x52/0xc0
kernel: [26339.017590] [<ffffffff813e3298>] ? cpuidle_enter_state+0x48/0xc0
kernel: [26339.017592] [<ffffffff810a9b28>] ? cpu_startup_entry+0x328/0x470
kernel: [26339.017595] [<ffffffff81043fdf>] ? start_secondary+0x20f/0x2d0
[....]
kernel: [26339.017647] swapper/13: page allocation failure: order:0, mode:0x20
kernel: [26339.017667] active_anon:2860787 inactive_anon:290478 isolated_anon:15723
kernel: [26339.017667] active_file:284318 inactive_file:151176 isolated_file:0
kernel: [26339.017667] unevictable:20736 dirty:24804 writeback:4297 unstable:0
kernel: [26339.017667] free:23079 slab_reclaimable:27293 slab_unreclaimable:86672
kernel: [26339.017667] mapped:22343 shmem:413 pagetables:10111 bounce:0
kernel: [26339.017667] free_cma:0
kernel: [26339.017670] Node 0 DMA free:15896kB min:64kB low:80kB high:96kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15980kB managed:15896kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
kernel: [26339.017675] lowmem_reserve[]: 0 3191 16016 16016
kernel: [26339.017680] Node 0 DMA32 free:56312kB min:13456kB low:16820kB high:20184kB active_anon:589468kB inactive_anon:141384kB active_file:1132312kB inactive_file:597576kB unevictable:16616kB isolated(anon):0kB isolated(file):0kB present:3345344kB managed:3270860kB mlocked:16616kB dirty:33860kB writeback:4288kB mapped:18616kB shmem:180kB slab_reclaimable:17036kB slab_unreclaimable:83696kB kernel_stack:34016kB pagetables:8384kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
kernel: [26339.017686] lowmem_reserve[]: 0 0 12824 12824
kernel: [26339.017691] Node 0 Normal free:20108kB min:54060kB low:67572kB high:81088kB active_anon:10853680kB inactive_anon:1020528kB active_file:4960kB inactive_file:7128kB unevictable:66328kB isolated(anon):62892kB isolated(file):0kB present:13369344kB managed:13131968kB mlocked:66328kB dirty:65356kB writeback:12900kB mapped:70756kB shmem:1472kB slab_reclaimable:92136kB slab_unreclaimable:262992kB kernel_stack:10880kB pagetables:32060kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:4275 all_unreclaimable? no
kernel: [26339.017696] lowmem_reserve[]: 0 0 0 0
kernel: [26339.017701] Node 0 DMA: 0*4kB 0000000000000020 ffff88042f1a3bf0
kernel: [26339.017706] 1*8kB (U) 1*16kB (U) 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15896kB
kernel: [26339.017723] Node 0 DMA32: 250*4kB
kernel: [26339.017726] ffffffff81144a3f 0000000000000000 0000000000000000 ffffffff00000002
kernel: [26339.017730] (EM) 967*8kB (UEM) 2628*16kB (UM) 83*32kB (UMR) 15*64kB (R) 8*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB <4>[26339.017747] swapper/0: page allocation failure: order:0, mode:0x20
kernel: [26339.017748] 0*4096kB = 56448kB
kernel: [26339.017751] Node 0 Normal: 3653*4kB (M) 0*8kB 0*16kB 1*32kB (R) 0*64kB 1*128kB (R) 0*256kB 1*512kB (R) 0*1024kB 1*2048kB (R) 0*4096kB = 17332kB
kernel: [26339.017767] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
kernel: [26339.017768] 466495 total pagecache pages
kernel: [26339.017769] 10046 pages in swap cache
kernel: [26339.017771] Swap cache stats: add 4415081, delete 4405035, find 1682225/2488531
kernel: [26339.017772] Free swap = 19301256kB
kernel: [26339.017773] Total swap = 19764220kB
kernel: [26339.017774] 4182667 pages RAM
kernel: [26339.017775] 0 pages HighMem/MovableOnly
kernel: [26339.017776] 59344 pages reserved
kernel: [26339.017777] 0 pages hwpoisoned
Kernel panic show messages related about irq and ixgbe.
Could someone give me some advice to solve this problem ? The server was running fine during 2 hours with same network load without any problems.
Regards,

The call trace cannot show any debug information related to kernel panic.
kernel: [26339.017509] [<ffffffff81144a3f>] ? warn_alloc_failed+0xdf/0x130
kernel: [26339.017513] [<ffffffff810a949d>] ? __wake_up_sync_key+0x3d/0x60
kernel: [26339.017515] [<ffffffff81148daf>] ? __alloc_pages_nodemask+0x8ef/0xb50
kernel: [26339.017519] [<ffffffff8147eaff>] ? tcp_v4_do_rcv+0x1af/0x4c0
kernel: [26339.017524] [<ffffffff81455b66>] ? nf_hook_slow+0x76/0x130
kernel: [26339.017528] [<ffffffff811883ad>] ? alloc_pages_current+0x9d/0x150
kernel: [26339.017531] [<ffffffff81412d7b>] ? __netdev_alloc_frag+0x8b/0x140
kernel: [26339.017534] [<ffffffff8141913f>] ? __netdev_alloc_skb+0x6f/0xf0
kernel: [26339.017558] [<ffffffffa0146a0d>] ? ixgbe_clean_rx_irq+0x10d/0xb70 [ixgbe]
kernel: [26339.017564] [<ffffffffa0148198>] ? ixgbe_poll+0x488/0x860 [ixgbe]
kernel: [26339.017567] [<ffffffff8108c9ad>] ? hrtimer_get_next_event+0xad/0xc0
Rather than above call trace, below signature indicates the sign of page starvation.
kernel: [26339.017667] active_anon:2860787 inactive_anon:290478 isolated_anon:15723
kernel: [26339.017667] active_file:284318 inactive_file:151176 isolated_file:0
kernel: [26339.017667] unevictable:20736 dirty:24804 writeback:4297 unstable:0
kernel: [26339.017667] free:23079 slab_reclaimable:27293 slab_unreclaimable:86672
kernel: [26339.017667] mapped:22343 shmem:413 pagetables:10111 bounce:0
kernel: [26339.017667] free_cma:0
As the "inactive_anon:290478, inactive_file:151176" signature indicates, there a high possibility of DMA zone page starvation.
If you refer to the following instruction, you find out whether our system is going through kernel memory leak.
Kernel: Add config associated with kmem leak
diff --git a/arch/arm/configs/pompeii_defconfig b/arch/arm/configs/pompeii_defconfig
index 2e97f97..aac678a 100644
--- a/arch/arm/configs/pompeii_defconfig
+++ b/arch/arm/configs/pompeii_defconfig
## -754,8 +754,8 ##
CONFIG_SLUB_DEBUG_PANIC_ON=y
CONFIG_SLUB_DEBUG_ON=y
CONFIG_DEBUG_KMEMLEAK=y
-CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=4000
-CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF=y
+CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=40000
+# CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF is not set
CONFIG_DEBUG_STACK_USAGE=y
CONFIG_DEBUG_VM=y
CONFIG_DEBUG_MEMORY_INIT=y
Make sure to add kernel command line to "" kmemleak=on"".
After typing the following command for 10 mins period,
echo scan > /sys/kernel/debug/kmemleak
The output of kernel memory leak can be displayed with below command.
cat > /sys/kernel/debug/kmemleak

Related

Unable to run linux 3.10(mips) on qemu 2.5

I want to run linux 3.10 with mips64r2 on qemu. But it fails, the boot log as follows,
I compile the kernel with the gcc 4.9.3 which is modified by loongson.
The kernel config file is the malta_defconfig and i change it to mips64r2 cpu and 64 bit kernel.
The qemu 2.5 is the default application on the ubuntu 16.04.
zlp#lab302i-ES:~/projs/linux-3.10$ qemu-system-mips64el -M malta -m 1G -cpu 5KEf -kernel vmlinux -nographic
Linux version 3.10.0 (zlp#lab302i-ES) (gcc version 4.9.3 20150626 (Red Hat 4.9.3-2) (GCC) ) #8 SMP Tue Nov 19 19:16:32 CST 2019
Config serial console: console=ttyS0,38400n8r
bootconsole [early0] enabled
CPU revision is: 00018900 (MIPS 5KE)
FPU revision is: 00738900
Checking for the multiply/shift bug... no.
Checking for the daddiu bug... no.
Software DMA cache coherency enabled
Determined physical RAM map:
memory: 0000000000001000 # 0000000000000000 (reserved)
memory: 00000000000ef000 # 0000000000001000 (ROM data)
memory: 0000000000539000 # 00000000000f0000 (reserved)
memory: 000000000f9d7000 # 0000000000629000 (usable)
Wasting 88312 bytes for tracking 1577 unused pages
Zone ranges:
DMA [mem 0x00000000-0x00ffffff]
Normal [mem 0x01000000-0x0fffffff]
Movable zone start for each node
Early memory node ranges
node 0: [mem 0x00000000-0x0fffffff]
Primary instruction cache 8kB, VIPT, 2-way, linesize 32 bytes.
Primary data cache 8kB, 2-way, VIPT, no aliases, linesize 32 bytes
PERCPU: Embedded 10 pages/cpu #9800000001384000 s10816 r8192 d21952 u40960
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 64640
Kernel command line: console=ttyS0,38400n8r
PID hash table entries: 1024 (order: 1, 8192 bytes)
Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
Inode-cache hash table entries: 16384 (order: 5, 131072 bytes)
Cache parity protection disabled
Memory: 251796k/255836k available (3695k kernel code, 4040k reserved, 1150k data, 272k init, 0k highmem)
Hierarchical RCU implementation.
CONFIG_RCU_FANOUT set to non-default value of 32
RCU restricting CPUs from NR_CPUS=2 to nr_cpu_ids=1.
NR_IRQS:256
CPU frequency 200.00 MHz
Console: colour dummy device 80x25
Calibrating delay loop... 1076.42 BogoMIPS (lpj=5382144)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 256
Checking for the daddi bug... no.
Brought up 1 CPUs
NET: Registered protocol family 16
bio: create slab <bio-0> at 0
vgaarb: loaded
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x10000000-0x17ffffff]
pci_bus 0000:00: root bus resource [io 0x2000-0x1fffff]
pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
pci 0000:00:0a.3: no compatible bridge window for [io 0x1100-0x110f]
vgaarb: device added: PCI:0000:00:12.0,decodes=io+mem,owns=none,locks=none
pci 0000:00:0a.3: BAR 8: [io 0x1100-0x110f] has bogus alignment
pci 0000:00:12.0: BAR 0: assigned [mem 0x10000000-0x11ffffff pref]
pci 0000:00:0b.0: BAR 6: assigned [mem 0x12000000-0x1203ffff pref]
pci 0000:00:12.0: BAR 6: assigned [mem 0x12040000-0x1204ffff pref]
pci 0000:00:12.0: BAR 1: assigned [mem 0x12050000-0x12050fff]
pci 0000:00:0a.2: BAR 4: assigned [io 0x2000-0x201f]
pci 0000:00:0b.0: BAR 0: assigned [io 0x2020-0x203f]
pci 0000:00:0b.0: BAR 1: assigned [mem 0x12051000-0x1205101f]
pci 0000:00:0a.1: BAR 4: assigned [io 0x2040-0x204f]
Switching to clocksource pit
NET: Registered protocol family 2
TCP established hash table entries: 2048 (order: 3, 32768 bytes)
TCP bind hash table entries: 2048 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
TCP: reno registered
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
PCI: Enabling device 0000:00:0a.2 (0000 -> 0001)
CPU 0 Unable to handle kernel paging request at virtual address 0000000000000000, epc == ffffffff8016bef4, ra == ffffffff805c51a0
Oops[#1]:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.10.0 #8
task: 980000000fc433f0 ti: 980000000fc44000 task.ti: 980000000fc44000
$ 0 : 0000000000000000 0000000000000008 0000000000000000 0000000000000000
$ 4 : 980000000fc47de0 0000000000000000 0000000000000000 0000000000000000
$ 8 : 0000000000000000 980000000fd84e60 fffffffffffffffc ffffffff8061cd30
$12 : 0000000000000010 ffffffff802e3bd4 0000000000000000 000000000000001a
$16 : ffffffff80600000 ffffffff805c5184 0000000000000000 ffffffff80600000
$20 : ffffffff805e6368 ffffffff805e6338 ffffffff805bc1d8 ffffffff805e62f8
$24 : 0000000000000018 ffffffff803451b0
$28 : 980000000fc44000 980000000fc47de0 ffffffff80600000 ffffffff805c51a0
Hi : 0000000000000001
Lo : 1111111111111112
epc : ffffffff8016bef4 cmpxchg_futex_value_locked+0x2c/0x78
Not tainted
ra : ffffffff805c51a0 futex_init+0x1c/0x6c
Status: 1400a7e3 KX SX UX KERNEL EXL IE
Cause : 00800008
BadVA : 0000000000000000
PrId : 00018900 (MIPS 5KE)
Modules linked in:
Process swapper/0 (pid: 1, threadinfo=980000000fc44000, task=980000000fc433f0, tls=0000000000000000)
Stack : ffffffff805c4f3c 0000000000000000 ffffffff80600000 ffffffff801004f0
ffffffff805e6368 0000000000000006 0000000000000030 ffffffff805f0a30
ffffffff80600000 ffffffff805bca24 0000000000000066 0000000000000000
ffffffff80494a48 0000000000000000 ffffffff80600000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 ffffffff80494a64 ffffffff80494a48 0000000000000000
0000000000000000 ffffffff80101f18 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
...
Call Trace:
[<ffffffff8016bef4>] cmpxchg_futex_value_locked+0x2c/0x78
[<ffffffff805c51a0>] futex_init+0x1c/0x6c
[<ffffffff801004f0>] do_one_initcall+0xe0/0x160
[<ffffffff805bca24>] kernel_init_freeable+0x16c/0x220
[<ffffffff80494a64>] kernel_init+0x1c/0x160
[<ffffffff80101f18>] ret_from_kernel_thread+0x14/0x1c
Code: 00000000 0000102d 0000000f <c0a30000> 14660005 00000000 00e0082d e0a10000 1020fff9
---[ end trace 47a33b7db369802c ]---
note: swapper/0[1] exited with preempt_count 1
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
Several possible issues here:
(1) QEMU 2.5 is now pretty old. You could retry with something more modern.
(2) You're building the kernel with "a gcc modified by loongson", but you're not actually running this on a Loongson CPU. Try using a stock gcc to build your kernel, and make sure that your kernel config matches the board model and CPU model you're asking QEMU to emulate. (The kernel log you give shows it crashing in a low level function which is going to try to do an atomic cmpxchg operation, and I have a vague recollection that this is an area where different MIPS CPUs have different sets of supported instructions, and in particular that Loongson might have made some changes here. So my first guess would be that your problem is here.)

What causes "no page found" oops

I'm hitting following oops in the kfifo_unused() code. This issue is timing dependent as it does not occur it kernel debug messages are disabled.
Oops flags [ 82.848904] Oops: 0000 [#1] PREEMPT SMP indicate it's caused by "no page found". What can cause this type of error?
EDIT: I just noticed this line PGD 0. It seems that Page Global Directory is null. What gives?
[ 82.827981] BUG: unable to handle kernel
[ 82.832507] NULL pointer dereference at 000000000000001c
[ 82.838470] IP:
[ 82.840537] [<ffffffff8126b126>] kfifo_unused+0x0/0x10
PGD 0
[ 82.847245]
[ 82.848904] Oops: 0000 [#1] PREEMPT SMP
[ 82.853244] Modules linked in:
[ 82.856703] dal_gate portmux_intel_drcfg mwifiex_pcie mwifiex xhci_pci xhci_hcd cfg80211 ahci spidev usbcore x86_pkg_temp_thermal libahci libata sbi_apl usb_common pwm_lpss_pci tpm_crb spi_pxa2xx_platform pwm_lpss scsi_mod dwc3_pci tpm_tis tpm_tis_core evdev tpm aesni_intel ablk_helper lrw cryptd gf128mul aes_x86_64 glue_helper mei_dal ccm gcm crc32c_intel mei_me ctr ecb cbc mei snd_soc_skl snd_soc_tdf8532 snd_soc_bxt_gpmrb snd_hda_ext_core snd_soc_core snd_hda_core snd_soc_skl_ipc snd_soc_sst_dsp snd_soc_sst_ipc snd_aloop snd_pcm snd_compress snd_timer i915 rtc_cmos drm_kms_helper intel_gtt i2c_dev ipv6 crc_ccitt autofs4
[ 82.919279] CPU: 1 PID: 18 Comm: kworker/1:0 Not tainted 4.9.78-abl-11621-g109ff90 #4
[ 82.928085] Workqueue: events mei_cl_bus_rx_work [mei]
[ 82.933881] task: ffff88027eb80fc0 task.stack: ffffc900000b0000
[ 82.940551] RIP: 0010:[<ffffffff8126b126>]
[ 82.945273] [<ffffffff8126b126>] kfifo_unused+0x0/0x10
[ 82.951123] RSP: 0018:ffffc900000b3dc0 EFLAGS: 00010246
[ 82.957111] RAX: 0000000000000001 RBX: ffff880274c6cc00 RCX: 0000000000000018
[ 82.965147] RDX: 0000000000000008 RSI: ffff880274c6cfb0 RDI: 0000000000000018
[ 82.973180] RBP: ffffc900000b3dc8 R08: 0000000000000009 R09: 0000000000000002
[ 82.981209] R10: ffffc90000213d28 R11: 0000000000000018 R12: 0000000000000000
[ 82.989241] R13: 0000000000000020 R14: ffff880274c6cf28 R15: 0000000000000000
[ 82.997276] FS: 0000000000000000(0000) GS:ffff88027fa80000(0000) knlGS:0000000000000000
[ 83.006431] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 83.012903] CR2: 000000000000001c CR3: 0000000001a08000 CR4: 0000000000340670
[ 83.020932] Stack:
[ 83.023228] ffffffff8126b315
[ 83.026652] ffffc900000b3e08 ffffffffa021d1f5 0000000000000018
[ 83.033116] ffff8802762a8018
[ 83.036481] ffff880274027f00 ffff880274027f00 0000000000000000
[ 83.042946] ffffe8ffffc81c00
[ 83.046308] ffffc900000b3e28 ffffffffa0063ed6 ffff88027eaecf00
[ 83.052765] Call Trace:
[ 83.055548] [<ffffffff8126b315>] ? __kfifo_in+0xc/0x23
[ 83.061445] [<ffffffffa021d1f5>] dal_recv_cb+0xe5/0x18a [mei_dal]
So, looking closer at dal_recv_cb() turned out in finding an if-else {} setting up a pointer to dal client structure for non existing user-space client. In case of erroneous response, which could no be matched with current dal client, the user-space client was selected by default to handle the message. And since it was not set up, its pointer was null, which was not checked for before calling __kfifo_in().

Kernel panic on linux-omap 4.3.0-rc5 on IGEPv2 board

I'm trying to run the latest linux-omap kernel Linux-4.3.0-rc5-12674-g80923cf8e that I got from the official linux-omap GIT.
I compiled it with the multi_v7_defconfig, disabling unneeded boards and adding some debug options like earlyprintk.
I am now stuck with a kernel panic and I can't find what the problem is.
The board is IGEPv2 and I am using the omap3-igep0020.dts device tree file.
Kernel is loaded using the latest u-boot, if it helps.
Here's the bootlog. Any idea ?
## Booting kernel from Legacy Image at 80008000 ...
Image Name: Linux-4.3.0-rc5-12674-g80923cf8e
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 4945528 Bytes = 4.7 MiB
Load Address: 80008000
Entry Point: 80008000
Verifying Checksum ... OK
## Flattened Device Tree blob at 81f00000
Booting using the fdt blob at 0x81f00000
Loading Kernel Image ... OK
Loading Device Tree to 8ffec000, end 8ffff0a2 ... OK
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.3.0-rc5-12674-g80923cf8eb3b (b#b43) (gcc version 7.2.0 (Ubuntu/Linaro 7.2.0-6ubuntu1) ) #2 SMP Wed Feb 28 00:49:54 EST 2018
[ 0.000000] CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c5387d
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
[ 0.000000] Machine model: IGEPv2 Rev. C (TI OMAP AM/DM37x)
[ 0.000000] bootconsole [earlycon0] enabled
[ 0.000000] cma: Reserved 64 MiB at 0x9b800000
[ 0.000000] Memory policy: Data cache writeback
[ 0.000000] On node 0 totalpages: 130816
[ 0.000000] free_area_init_node: node 0, pgdat c09aee00, node_mem_map dfad9000
[ 0.000000] Normal zone: 1024 pages used for memmap
[ 0.000000] Normal zone: 0 pages reserved
[ 0.000000] Normal zone: 130816 pages, LIFO batch:31
[ 0.000000] CPU: All CPU(s) started in SVC mode.
[ 0.000000] OMAP3430/3530 ES3.1 (l2cache iva sgx neon isp )
[ 0.000000] PERCPU: Embedded 12 pages/cpu #dfa86000 s18572 r8192 d22388 u49152
[ 0.000000] pcpu-alloc: s18572 r8192 d22388 u49152 alloc=12*4096
[ 0.000000] pcpu-alloc: [0] 0
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 129792
[ 0.000000] Kernel command line: mem=512M console=ttyO2,115200n8 debug earlyprintk omapfb.mode=dvi:1024x768MR-16#60 root=/dev/mmcblk0p2 rw rootwait
[ 0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
[ 0.000000] Memory: 442476K/523264K available (6856K kernel code, 370K rwdata, 2264K rodata, 396K init, 295K bss, 15252K reserved, 65536K cma-reserved, 0K highmem)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
[ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
[ 0.000000] vmalloc : 0xe0800000 - 0xff000000 ( 488 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xe0000000 ( 512 MB)
[ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
[ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
[ 0.000000] .text : 0xc0008000 - 0xc08f0590 (9122 kB)
[ 0.000000] .init : 0xc08f1000 - 0xc0954000 ( 396 kB)
[ 0.000000] .data : 0xc0954000 - 0xc09b0980 ( 371 kB)
[ 0.000000] .bss : 0xc09b3000 - 0xc09fcc0c ( 296 kB)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] ⇥ Build-time adjustment of leaf fanout to 32.
[ 0.000000] ⇥ RCU restricting CPUs from NR_CPUS=16 to nr_cpu_ids=1.
[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=32, nr_cpu_ids=1
[ 0.000000] NR_IRQS:16 nr_irqs:16 16
[ 0.000000] IRQ: Found an INTC at 0xfa200000 (revision 4.0) with 96 interrupts
[ 0.000000] Clocking rate (Crystal/Core/MPU): 26.0/332/600 MHz
[ 0.000000] OMAP clockevent source: timer1 at 32768 Hz
[ 0.000000] clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns
[ 0.000000] sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps every 65535999984741ns
[ 0.008697] OMAP clocksource: 32k_counter at 32768 Hz
[ 0.014770] Console: colour dummy device 80x30
[ 0.019439] Calibrating delay loop... 597.60 BogoMIPS (lpj=2988032)
[ 0.084869] pid_max: default: 32768 minimum: 301
[ 0.089874] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.096801] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.105255] CPU: Testing write buffer coherency: ok
[ 0.110809] CPU0: thread -1, cpu 0, socket -1, mpidr 0
[ 0.116394] Setting up static identity map for 0x80008340 - 0x80008398
[ 0.124786] Brought up 1 CPUs
[ 0.127929] SMP: Total of 1 processors activated (597.60 BogoMIPS).
[ 0.134521] CPU: All CPU(s) started in SVC mode.
[ 0.141571] devtmpfs: initialized
[ 0.190948] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 1
[ 0.221343] omap_hwmod: mcbsp2_sidetone using broken dt data from mcbsp
[ 0.229003] omap_hwmod: mcbsp3_sidetone using broken dt data from mcbsp
[ 0.296447] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.311370] pinctrl core: initialized pinctrl subsystem
[ 0.319793] NET: Registered protocol family 16
[ 0.328643] DMA: preallocated 256 KiB pool for atomic coherent allocations
[ 0.338317] cpuidle: using governor ladder
[ 0.342712] cpuidle: using governor menu
[ 0.347473] Reprogramming SDRC clock to 332000000 Hz
[ 0.360107] OMAP GPIO hardware version 2.5
[ 0.385833] omap-gpmc 6e000000.gpmc: GPMC revision 5.0
[ 0.391235] gpmc_mem_init: disabling cs 0 mapped at 0x0-0x1000000
[ 0.400299] platform 49042000.serial: Cannot lookup hwmod 'uart4'
[ 0.412506] No ATAGs?
[ 0.414703] hw-breakpoint: debug architecture 0x4 unsupported.
[ 0.422790] OMAP DMA hardware revision 4.0
[ 0.427307] Serial: AMBA PL011 UART driver
[ 0.458862] omap-dma-engine 48056000.dma-controller: OMAP DMA engine driver
[ 0.474639] vgaarb: loaded
[ 0.478942] SCSI subsystem initialized
[ 0.483337] libata version 3.00 loaded.
[ 0.488037] usbcore: registered new interface driver usbfs
[ 0.493957] usbcore: registered new interface driver hub
[ 0.499603] usbcore: registered new device driver usb
[ 0.506896] omap_i2c 48070000.i2c: could not find pctldev for node /ocp/l4#48000000/scm#2000/pinmux#30/pinmux_i2c1_pins, deferring probe
[ 0.520629] omap_i2c 48072000.i2c: bus 1 rev3.3 at 100 kHz
[ 0.526519] omap_i2c 48060000.i2c: could not find pctldev for node /ocp/l4#48000000/scm#2000/pinmux#30/pinmux_i2c3_pins, deferring probe
[ 0.539611] pps_core: LinuxPPS API ver. 1 registered
[ 0.544860] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti#linux.it>
[ 0.554412] PTP clock support registered
[ 0.558990] EDAC MC: Ver: 3.0.0
[ 0.565734] clocksource: Switched to clocksource 32k_counter
[ 0.593200] NET: Registered protocol family 2
[ 0.598846] TCP established hash table entries: 4096 (order: 2, 16384 bytes)
[ 0.606323] TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.613128] TCP: Hash tables configured (established 4096 bind 4096)
[ 0.619903] UDP hash table entries: 256 (order: 1, 8192 bytes)
[ 0.626068] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[ 0.632873] NET: Registered protocol family 1
[ 0.638061] RPC: Registered named UNIX socket transport module.
[ 0.644226] RPC: Registered udp transport module.
[ 0.649230] RPC: Registered tcp transport module.
[ 0.654144] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 0.660919] PCI: CLS 0 bytes, default 64
[ 0.668243] hw perfevents: enabled with armv7_cortex_a8 PMU driver, 5 counters available
[ 0.680969] futex hash table entries: 256 (order: 2, 16384 bytes)
[ 0.707794] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 0.715759] NFS: Registering the id_resolver key type
[ 0.721191] Key type id_resolver registered
[ 0.725585] Key type id_legacy registered
[ 0.729949] ntfs: driver 2.1.32 [Flags: R/O].
[ 0.735290] jffs2: version 2.2. (NAND) <0xc2><0xa9> 2001-2006 Red Hat, Inc.
[ 0.743438] fuse init (API version 7.23)
[ 0.750671] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
[ 0.758483] io scheduler noop registered
[ 0.762634] io scheduler deadline registered
[ 0.767242] io scheduler cfq registered (default)
[ 0.776153] pinctrl-single 48002030.pinmux: 284 pins at pa fa002030 size 568
[ 0.783874] pinctrl-single 48002a00.pinmux: 46 pins at pa fa002a00 size 92
[ 0.791748] Unhandled fault: external abort on non-linefetch (0x1028) at 0xfa0025a0
[ 0.799713] pgd = c0004000
[ 0.802581] [fa0025a0] *pgd=48011452(bad)
[ 0.806823] Internal error: : 1028 [#1] SMP ARM
[ 0.811553] Modules linked in:
[ 0.814819] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.3.0-rc5-12674-g80923cf8eb3b #2
[ 0.823028] Hardware name: Generic OMAP36xx (Flattened Device Tree)
[ 0.829589] task: db074000 ti: db078000 task.ti: db078000
[ 0.835235] PC is at pcs_readw+0x0/0x10
[ 0.839294] LR is at pcs_probe+0x388/0x804
[ 0.843597] pc : [<c02be958>] lr : [<c02c0700>] psr: 20000153
[ 0.843597] sp : db079de0 ip : 00000000 fp : 0000002e
[ 0.855560] r10: 00000000 r9 : 00000002 r8 : 00000000
[ 0.861053] r7 : 00000001 r6 : 00000000 r5 : 00000002 r4 : db2d4e10
[ 0.867828] r3 : c02be958 r2 : db21fc00 r1 : 00000002 r0 : fa0025a0
[ 0.874664] Flags: nzCv IRQs on FIQs off Mode SVC_32 ISA ARM Segment none
[ 0.882171] Control: 10c5387d Table: 80004019 DAC: 00000051
[ 0.888183] Process swapper/0 (pid: 1, stack limit = 0xdb078218)
[ 0.894439] Stack: (0xdb079de0 to 0xdb07a000)
[ 0.899017] 9de0: c0824d64 dfac90a0 c0974cd8 00000000 00000002 dfac90a0 0000000c c0824fcc
[ 0.907531] 9e00: db2d4ea8 00000000 db161c00 00000000 db2d4e40 db2d4e28 db2da9b0 db2da9b0
[ 0.916046] 9e20: c09b082f c0156d78 db2da9b0 00000000 00000000 c08302ec 00000001 c0156f34
[ 0.924560] 9e40: db163cd0 c08302ec 00000001 000000ec db163cd0 db2da7d0 db2da9b0 db163cd0
[ 0.933074] 9e60: c08302ec db161c10 ffffffed c0974cd8 fffffdfb 00000000 000000ec c094d048
[ 0.941589] 9e80: 00000000 c038b544 c09ec6ec db161c10 c0974cd8 00000000 00000000 c0389e2c
[ 0.950103] 9ea0: db161c10 c0974cd8 db161c44 00000000 00000000 c0389f9c 00000000 c0974cd8
[ 0.958587] 9ec0: c0389f18 c03882d0 db00cb5c db162434 c0974cd8 db2d9780 c0987578 c038940c
[ 0.967102] 9ee0: c0824d64 c09b3000 c0974cd8 c0974cd8 c0959bb0 db2db240 c0912288 c038a68c
[ 0.975616] 9f00: c0959bb0 c0959bb0 db2db240 c000985c 00000000 c014e320 c014ddd8 db11ab80
[ 0.984130] 9f20: 00000000 c096a858 00000000 c06aad30 dfeff132 c00504d0 cccccccd c08bebb8
[ 0.992645] 9f40: c085f958 00000006 00000006 c07f9410 c096a840 c09b3000 c09b3000 c0939830
[ 1.001159] 9f60: c09b3000 c09b3000 c0939834 c093983c 00000007 c08f1dc0 00000006 00000006
[ 1.009674] 9f80: 00000000 c08f159c 00000000 c0683b30 00000000 00000000 00000000 00000000
[ 1.018188] 9fa0: 00000000 c0683b38 00000000 c000f638 00000000 00000000 00000000 00000000
[ 1.026702] 9fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 1.035186] 9fe0: 00000000 00000000 00000000 00000000 00000013 00000000 fdfeffff f7ff777f
[ 1.043731] [<c02be958>] (pcs_readw) from [<c02c0700>] (pcs_probe+0x388/0x804)
[ 1.051269] [<c02c0700>] (pcs_probe) from [<c038b544>] (platform_drv_probe+0x48/0xa0)
[ 1.059417] [<c038b544>] (platform_drv_probe) from [<c0389e2c>] (driver_probe_device+0x1ec/0x2d8)
[ 1.068664] [<c0389e2c>] (driver_probe_device) from [<c0389f9c>] (__driver_attach+0x84/0x88)
[ 1.077423] [<c0389f9c>] (__driver_attach) from [<c03882d0>] (bus_for_each_dev+0x4c/0x9c)
[ 1.085937] [<c03882d0>] (bus_for_each_dev) from [<c038940c>] (bus_add_driver+0x18c/0x210)
[ 1.094543] [<c038940c>] (bus_add_driver) from [<c038a68c>] (driver_register+0x78/0xf4)
[ 1.102874] [<c038a68c>] (driver_register) from [<c000985c>] (do_one_initcall+0x8c/0x1d8)
[ 1.111389] [<c000985c>] (do_one_initcall) from [<c08f1dc0>] (kernel_init_freeable+0x158/0x1e8)
[ 1.120452] [<c08f1dc0>] (kernel_init_freeable) from [<c0683b38>] (kernel_init+0x8/0xe8)
[ 1.128875] [<c0683b38>] (kernel_init) from [<c000f638>] (ret_from_fork+0x14/0x3c)
[ 1.136779] Code: e5d00000 e6ef0070 f57ff04f e12fff1e (e1d000b0)
[ 1.143157] ---[ end trace 51c6d6f40dd2a6d2 ]---
[ 1.148040] In-band Error seen by MPU at address 0
[ 1.153167] ------------[ cut here ]------------
[ 1.158020] WARNING: CPU: 0 PID: 1 at drivers/bus/omap_l3_smx.c:166 omap3_l3_app_irq+0xc4/0x118()
[ 1.167236] Modules linked in:
[ 1.170471] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G D 4.3.0-rc5-12674-g80923cf8eb3b #2
[ 1.179962] Hardware name: Generic OMAP36xx (Flattened Device Tree)
[ 1.186523] [<c00174f0>] (unwind_backtrace) from [<c0012ff4>] (show_stack+0x10/0x14)
[ 1.194580] [<c0012ff4>] (show_stack) from [<c028eea8>] (dump_stack+0x80/0x90)
[ 1.202117] [<c028eea8>] (dump_stack) from [<c00365f0>] (warn_slowpath_common+0x80/0xb0)
[ 1.210540] [<c00365f0>] (warn_slowpath_common) from [<c00366bc>] (warn_slowpath_null+0x1c/0x24)
[ 1.219665] [<c00366bc>] (warn_slowpath_null) from [<c02b5ec0>] (omap3_l3_app_irq+0xc4/0x118)
[ 1.228546] [<c02b5ec0>] (omap3_l3_app_irq) from [<c00787e0>] (handle_irq_event_percpu+0x48/0x138)
[ 1.237884] [<c00787e0>] (handle_irq_event_percpu) from [<c0078918>] (handle_irq_event+0x48/0x6c)
[ 1.247100] [<c0078918>] (handle_irq_event) from [<c007b584>] (handle_level_irq+0xac/0x144)
[ 1.255798] [<c007b584>] (handle_level_irq) from [<c0077f24>] (generic_handle_irq+0x24/0x34)
[ 1.264587] [<c0077f24>] (generic_handle_irq) from [<c00781b0>] (__handle_domain_irq+0x5c/0xb4)
[ 1.273620] [<c00781b0>] (__handle_domain_irq) from [<c0013bd4>] (__irq_svc+0x54/0x70)
[ 1.281860] [<c0013bd4>] (__irq_svc) from [<c0038028>] (do_exit+0x3e0/0x988)
[ 1.289215] [<c0038028>] (do_exit) from [<c0013324>] (die+0x32c/0x330)
[ 1.296020] [<c0013324>] (die) from [<c0009380>] (do_DataAbort+0xb4/0xb8)
[ 1.303100] [<c0009380>] (do_DataAbort) from [<c0013b60>] (__dabt_svc+0x40/0x60)
[ 1.310821] Exception stack(0xdb079d90 to 0xdb079dd8)
[ 1.316101] 9d80: fa0025a0 00000002 db21fc00 c02be958
[ 1.324615] 9da0: db2d4e10 00000002 00000000 00000001 00000000 00000002 00000000 0000002e
[ 1.333129] 9dc0: 00000000 db079de0 c02c0700 c02be958 20000153 ffffffff
[ 1.340026] [<c0013b60>] (__dabt_svc) from [<c02be958>] (pcs_readw+0x0/0x10)
[ 1.347381] [<c02be958>] (pcs_readw) from [<c02c0700>] (pcs_probe+0x388/0x804)
[ 1.354888] [<c02c0700>] (pcs_probe) from [<c038b544>] (platform_drv_probe+0x48/0xa0)
[ 1.363067] [<c038b544>] (platform_drv_probe) from [<c0389e2c>] (driver_probe_device+0x1ec/0x2d8)
[ 1.372283] [<c0389e2c>] (driver_probe_device) from [<c0389f9c>] (__driver_attach+0x84/0x88)
[ 1.381072] [<c0389f9c>] (__driver_attach) from [<c03882d0>] (bus_for_each_dev+0x4c/0x9c)
[ 1.389587] [<c03882d0>] (bus_for_each_dev) from [<c038940c>] (bus_add_driver+0x18c/0x210)
[ 1.398193] [<c038940c>] (bus_add_driver) from [<c038a68c>] (driver_register+0x78/0xf4)
[ 1.406524] [<c038a68c>] (driver_register) from [<c000985c>] (do_one_initcall+0x8c/0x1d8)
[ 1.415039] [<c000985c>] (do_one_initcall) from [<c08f1dc0>] (kernel_init_freeable+0x158/0x1e8)
[ 1.424072] [<c08f1dc0>] (kernel_init_freeable) from [<c0683b38>] (kernel_init+0x8/0xe8)
[ 1.432495] [<c0683b38>] (kernel_init) from [<c000f638>] (ret_from_fork+0x14/0x3c)
[ 1.440368] ---[ end trace 51c6d6f40dd2a6d3 ]---
[ 1.445343] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 1.445343]
[ 1.454925] Rebooting in 300
I'm confused because there seem to be several stacktraces.
Also I wasn't able to get a more detailed stacktrace despite compiling the debug info in the kernel (but I suspect the debug info is not actually there as the file is surprisingly small)
The last supported kernel from the manufacturer ISEE for this board is 2.6.37 which is horribly old. There does not seem to be any problem with running more recent kernels but it's possible the device tree files have not been updated for long... I'm just looking for pointers of where this error might come from and I'll try to fix it.
Thanks !
I don't know what pcs is, but looks like it reads from wrong memory region.
[ 1.340026] [<c0013b60>] (__dabt_svc) from [<c02be958>] (pcs_readw+0x0/0x10)
[ 1.347381] [<c02be958>] (pcs_readw) from [<c02c0700>] (pcs_probe+0x388/0x804)
[ 1.354888] [<c02c0700>] (pcs_probe) from [<c038b544>] (platform_drv_probe+0x48/0xa0)
The reason why you getting page fault is an data abort in SVC (kernel) mode. That might happens because:
There is no any physical memory or IO space under physical addresses, which was specified in device tree for PCS driver. (possibly the used dtb is not compatible with the soc).
The memory which was accessed by pcs_readw is not mapped at all (or mapped with wrong attributes).
To figure out you need to check where virtual addresses, which are accessed by pcs_readw was mapped. What physical addresses are used. Check soc manual to be sure that this physical address fits in soc memory map.
Alex above nailed it !
I was writing an answer because I also found the problem in the meantime.
Basically, the board I am using, IGEPv2, was released in several revisions which use different CPUs. Apparently I have an early one (revision RB1) which uses an OMAP3530 CPU whereas the most recent ones are using a DM3730 CPU.
So, in the omap3-igep.dtsi file (used in the dts) I changed the include from omap36xx.dtsi to omap34xx.dtsi and boom, it boots successfully.
Wifi should not be working (because they changed the GPIO of the RESET pins of the Wifi and Bluetooth controller between the revisions) but now the system boots properly so it's just a matter of tracking the changes between the versions and editing the DTS files accordingly.
Hope this helps someone in the same boat - not sure if I'll be able to send these changes upstream so at least if someone Google the issue they'll find it !

XEN GUest - Atheros WiFi driver load causes memory paging failure

Some background -
We are trying to run Qualcomm Atheros AR928X Wireless Network Adapter and have a crash right on driver load, following are our observations and questions.
The Atheros card "Qualcomm Atheros AR928X Wireless Network Adapter (PCI-Express) (rev 01)" is plugged into the host system (datatron).
When I attach it to the DomU - the module "ath9k" is automatically loaded, but it gives an exception "iowrite32+0x2b/0x30".
No idea what the issue is (tried also with another Atheros Card (ath10k) - similar problem). When I try an Intel card, it works.
(the card also works on the Dom0 - so the Linux driver and HW is OK).
Debugging -
After some investigation with kgdb and iommu trace on DomU it seems the iomap of PCI BAR for the device returns a a mapping f which first 0x1000 bytes are read only and that causes access violation when trying to write registers mapped to this area (all the regs with offset < 0x1000) - why this happens i still don't know. Register writes with offsets > 0x1000 are fine.
Running same driver on Dom0 is totally fine
Bellow the sigsev backtrace and xen dmesg from DomU
As can be seen there is ioremap_ of size 0x10000 starting at ffffc900402c0000 but as i said, i noticed that anything bellow ffffc900402c1000 is not writable (from gdb using set addr = val) and only readable while anything above this address is writeable.
Question -
Please give any advise on this issue and especially how to approach debugging this both on Domu and Dom0 and where in xen code to look for possible issues.
[ 37.837467] ath9k 0000:00:00.0: Xen PCI mapped GSI16 to IRQ8
[ 37.837473] pcifront pci-0: read dev=0000:00:00.0 - offset 3d size 1
[ 37.837498] pcifront pci-0: read got back value 1
[ 37.837505] pcifront pci-0: read dev=0000:00:00.0 - offset 4 size 2
[ 37.840006] pcifront pci-0: read got back value 103
[ 37.853341] pcifront pci-0: read dev=0000:00:00.0 - offset c size 1
[ 37.853374] pcifront pci-0: read got back value 8
[ 37.853383] pcifront pci-0: write dev=0000:00:00.0 - offset d size 1 val a8
[ 37.853431] pcifront pci-0: read dev=0000:00:00.0 - offset 4 size 2
[ 37.853462] pcifront pci-0: read got back value 103
[ 37.853472] pcifront pci-0: write dev=0000:00:00.0 - offset 4 size 2 val 107
[ 37.853527] pcifront pci-0: read dev=0000:00:00.0 - offset 40 size 4
[ 37.853565] pcifront pci-0: read got back value 3c25001
[ 37.853574] pcifront pci-0: write dev=0000:00:00.0 - offset 40 size 4 val 3c20001
[ 37.855784] mmiotrace: ioremap_*(0xf7b00000, 0x10000) = ffffc900402c0000
[ 37.855842] pcifront pci-0: read dev=0000:00:00.0 - offset c size 1
[ 37.855879] pcifront pci-0: read got back value 8
[ 38.301919] BUG: unable to handle kernel paging request at ffffc900402c0040
[ 38.301930] IP: [<ffffffff8132387b>] iowrite32+0x2b/0x30
[ 38.301939] PGD 3fdf4067 PUD 3e1a8067 PMD 49d7067 PTE 80100000f7b00075
[ 38.301947] Oops: 0003 [#1] SMP
[ 38.301952] Modules linked in: ath9k(OE+) ath9k_common(OE) ath9k_hw(OE) ath(OE) mac80211(E) cfg80211(E) rfkill(E) xen_pcifront(E) intel_rapl(E) x86_pkg_temp_thermal(E) coretemp(E) crct10dif_pclmul(E) crc32_pclmul(E) evdev(E) ghash_clmulni_intel(E) pcspkr(E) uio_netx(E) uio(E) autofs4(E) ext4(E) ecb(E) crc16(E) mbcache(E) jbd2(E) crc32c_generic(E) crc32c_intel(E) aesni_intel(E) xen_netfront(E) xen_blkfront(E) aes_x86_64(E) glue_helper(E) lrw(E) gf128mul(E) ablk_helper(E) cryptd(E)
[ 38.302000] CPU: 0 PID: 696 Comm: systemd-udevd Tainted: G W OE 4.5.3-dbg #2
[ 38.302008] task: ffff8800044880c0 ti: ffff8800048a8000 task.ti: ffff8800048a8000
[ 38.302015] RIP: e030:[<ffffffff8132387b>] [<ffffffff8132387b>] iowrite32+0x2b/0x30
[ 38.302024] RSP: e02b:ffff8800048ab8c8 EFLAGS: 00010196
[ 38.302029] RAX: 0000000000000000 RBX: 000000000000000e RCX: ffff88003ac34018
[ 38.302035] RDX: ffffc900402c0040 RSI: ffffc900402c0040 RDI: 0000000000000000
[ 38.302040] RBP: ffff8800048ab928 R08: 0000000000000016 R09: 0000000000000002
[ 38.302046] R10: ffffffff81b08000 R11: ffffffff81b07fc0 R12: 0000000000000016
[ 38.302051] R13: ffff880004b4d098 R14: ffff8800048abec0 R15: ffffffffc03c74c0
[ 38.302060] FS: 00007ff256fb18c0(0000) GS:ffff88003f800000(0000) knlGS:ffff88003f800000
[ 38.302068] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 38.302073] CR2: ffffc900402c0040 CR3: 0000000004a05000 CR4: 0000000000042660
[ 38.302079] Stack:
[ 38.302083] ffffffffc03a70f5 0000000000000040 ffff88003ac34018 ffffffffc0339a5b
[ 38.302092] ffff88003ac34018 ffff88003ac34068 ffff88003ac81520 0000809400008090
[ 38.302102] 0000000000008098 0000000000000000 0000000000000000 000000003c5869cf
[ 38.302111] Call Trace:
[ 38.302123] [<ffffffffc03a70f5>] ? ath9k_iowrite32+0xde/0xf5 [ath9k]
[ 38.302139] [<ffffffffc0339a5b>] ? ath9k_hw_update_mibstats+0x62/0xd6 [ath9k_hw]
[ 38.302155] [<ffffffffc033a722>] ? ath9k_enable_mib_counters+0xc2/0x119 [ath9k_hw]
[ 38.302171] [<ffffffffc033a983>] ? ath9k_hw_ani_init+0x148/0x14b [ath9k_hw]
[ 38.302182] [<ffffffffc0313b00>] ? ath9k_hw_post_init+0x104/0x16c [ath9k_hw]
[ 38.302195] [<ffffffffc0313f22>] ? __ath9k_hw_init+0x377/0x3fd [ath9k_hw]
[ 38.302206] [<ffffffffc031403e>] ? ath9k_hw_init+0x96/0xd6 [ath9k_hw]
[ 38.302218] [<ffffffffc03a877a>] ? ath9k_init_softc+0x643/0x823 [ath9k]
[ 38.302229] [<ffffffffc03a8f7d>] ? ath9k_init_device+0x43/0x21d [ath9k]
[ 38.302243] [<ffffffffc03aad2e>] ? ath9k_tasklet+0x462/0x462 [ath9k]
[ 38.302255] [<ffffffffc03c1638>] ? ath_pci_probe+0x2c8/0x38f [ath9k]
[ 38.302263] [<ffffffff815bab16>] ? _raw_spin_unlock_irqrestore+0x16/0x20
[ 38.302271] [<ffffffff814431a4>] ? __pm_runtime_resume+0x54/0x70
[ 38.302278] [<ffffffff813566ff>] ? local_pci_probe+0x3f/0x90
[ 38.302284] [<ffffffff81357ab0>] ? pci_device_probe+0x100/0x140
[ 38.302290] [<ffffffff8143853c>] ? driver_probe_device+0x21c/0x430
[ 38.302296] [<ffffffff814387cb>] ? __driver_attach+0x7b/0x80
[ 38.302301] [<ffffffff81438750>] ? driver_probe_device+0x430/0x430
[ 38.302308] [<ffffffff81436007>] ? bus_for_each_dev+0x67/0xb0
[ 38.302313] [<ffffffff8143773f>] ? bus_add_driver+0x1df/0x270
[ 38.302319] [<ffffffffc02f0000>] ? 0xffffffffc02f0000
[ 38.302324] [<ffffffff81438fe7>] ? driver_register+0x57/0xc0
[ 38.302334] [<ffffffffc03c19ad>] ? ath_pci_init+0x23/0x25 [ath9k]
[ 38.302345] [<ffffffffc02f000d>] ? ath9k_init+0xd/0x1000 [ath9k]
[ 38.302352] [<ffffffff81002122>] ? do_one_initcall+0xb2/0x200
[ 38.302358] [<ffffffff815b74d4>] ? preempt_schedule_common+0x14/0x30
[ 38.302363] [<ffffffff815b7509>] ? _cond_resched+0x19/0x20
[ 38.302371] [<ffffffff811cf032>] ? kmem_cache_alloc_trace+0x82/0x220
[ 38.302378] [<ffffffff8116d58a>] ? do_init_module+0x5b/0x1ce
[ 38.302384] [<ffffffff810fa676>] ? load_module+0x2146/0x2790
[ 38.302389] [<ffffffff810f71d0>] ? __symbol_put+0x60/0x60
[ 38.302395] [<ffffffff810f7628>] ? copy_module_from_fd.isra.51+0xf8/0x140
[ 38.302401] [<ffffffff810faed8>] ? SYSC_finit_module+0xa8/0xd0
[ 38.302408] [<ffffffff815bb072>] ? system_call_fast_compare_end+0xc/0x67
[ 38.302413] Code: 48 81 fe ff ff 03 00 48 89 f2 77 1f 48 81 fe 00 00 01 00 76 07 0f b7 d6 89 f8 ef c3 48 c7 c6 df 6a 84 81 48 89 d7 e9 85 fe ff ff <89> 3e c3 66 90 48 81 ff ff ff 03 00 77 28 48 81 ff 00 00 01 00
[ 38.302460] RIP [<ffffffff8132387b>] iowrite32+0x2b/0x30
[ 38.302466] RSP <ffff8800048ab8c8>
[ 38.302469] CR2: ffffc900402c0040
[ 38.302475] ---[ end trace 092ecee276c2da16 ]---
(XEN) Xen version 4.6.0 (Debian 4.6.0-1+nmu2) (ijc#debian.org) (gcc (Debian 5.3.1-8) 5.3.1 20160205) debug=n Tue Feb 9 17:46:27 UTC 2016
(XEN) Bootloader: GRUB 2.02~beta2-36
(XEN) Command line: placeholder log_lvl=all loglvl=all guest_loglvl=all iommu=verbose
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN) EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN) Found 3 MBR signatures
(XEN) Found 3 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 0000000000099c00 (usable)
(XEN) 0000000000099c00 - 00000000000a0000 (reserved)
(XEN) 00000000000e0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 0000000020000000 (usable)
(XEN) 0000000020000000 - 0000000020200000 (reserved)
(XEN) 0000000020200000 - 0000000040004000 (usable)
(XEN) 0000000040004000 - 0000000040005000 (reserved)
(XEN) 0000000040005000 - 00000000d9d01000 (usable)
(XEN) 00000000d9d01000 - 00000000da191000 (reserved)
(XEN) 00000000da191000 - 00000000da192000 (ACPI data)
(XEN) 00000000da192000 - 00000000da2af000 (ACPI NVS)
(XEN) 00000000da2af000 - 00000000da6da000 (reserved)
(XEN) 00000000da6da000 - 00000000da6db000 (usable)
(XEN) 00000000da6db000 - 00000000da71e000 (ACPI NVS)
(XEN) 00000000da71e000 - 00000000daddf000 (usable)
(XEN) 00000000daddf000 - 00000000daff2000 (reserved)
(XEN) 00000000daff2000 - 00000000db000000 (usable)
(XEN) 00000000db800000 - 00000000dfa00000 (reserved)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec01000 (reserved)
(XEN) 00000000fed00000 - 00000000fed04000 (reserved)
(XEN) 00000000fed1c000 - 00000000fed20000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000ff000000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 000000041e600000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT DA2A0080, 007C (r1 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: FACP DA2A9C58, 00F4 (r4 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: DSDT DA2A0190, 9AC5 (r2 ALASKA A M I 1 INTL 20051117)
(XEN) ACPI: FACS DA2ADF80, 0040
(XEN) ACPI: APIC DA2A9D50, 0092 (r3 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: FPDT DA2A9DE8, 0044 (r1 ALASKA A M I 1072009 AMI 10013)
(XEN) ACPI: MCFG DA2A9E30, 003C (r1 ALASKA A M I 1072009 MSFT 97)
(XEN) ACPI: HPET DA2A9E70, 0038 (r1 ALASKA A M I 1072009 AMI. 5)
(XEN) ACPI: SSDT DA2A9EA8, 036D (r1 SataRe SataTabl 1000 INTL 20091112)
(XEN) ACPI: SSDT DA2AA218, 08A2 (r1 PmRef Cpu0Ist 3000 INTL 20051117)
(XEN) ACPI: SSDT DA2AAAC0, 0A92 (r1 PmRef CpuPm 3000 INTL 20051117)
(XEN) ACPI: DMAR DA2AB558, 00B8 (r1 INTEL SNB 1 INTL 1)
(XEN) ACPI: ASF! DA2AB610, 00A5 (r32 INTEL HCG 1 TFSM F4240)
(XEN) ACPI: BGRT DA2AB6B8, 0038 (r0 ALASKA A M I 1072009 AMI 10013)
(XEN) System RAM: 16263MB (16653732kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000041e600000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000fd7e0
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:404,1:0], pm1x_evt[1:400,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - da2adf80/0000000000000000, using 32
(XEN) ACPI: wakeup_vec[da2adf8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 7:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 7:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) Processor #4 7:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) Processor #6 7:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
(XEN) Processor #1 7:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
(XEN) Processor #3 7:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
(XEN) Processor #5 7:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
(XEN) Processor #7 7:10 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) ERST table was not found
(XEN) ACPI: BGRT: invalidating v1 image at 0xce4c7018
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) Intel machine check reporting enabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2292.620 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802bd010 -> ffff82d0802be240
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: MCFG area at f8000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 64 KiB.
(XEN) mwait-idle: MWAIT substates: 0x21120
(XEN) mwait-idle: v0.4 model 0x3a
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB
(XEN) Brought up 8 CPUs
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) Dom0 has maximum 792 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1d59000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 000000040e000000->0000000410000000 (4062057 pages to be allocated)
(XEN) Init. ramdisk: 000000041d4d6000->000000041e5ff665
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff81d59000
(XEN) Init. ramdisk: 0000000000000000->0000000000000000
(XEN) Phys-Mach map: 0000008000000000->0000008001f16498
(XEN) Start info: ffffffff81d59000->ffffffff81d594b4
(XEN) Page tables: ffffffff81d5a000->ffffffff81d6d000
(XEN) Boot stack: ffffffff81d6d000->ffffffff81d6e000
(XEN) TOTAL: ffffffff80000000->ffffffff82000000
(XEN) ENTRY ADDRESS: ffffffff81b2b1f0
(XEN) Dom0 has maximum 8 VCPUs
(XEN) Bogus DMIBAR 0xfed18001 on 0000:00:00.0
(XEN) Scrubbing Free RAM on 1 nodes using 4 CPUs
(XEN) .................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 288kB init memory.
(XEN) Bogus DMIBAR 0xfed18001 on 0000:00:00.0
(XEN) PCI add device 0000:00:00.0
(XEN) PCI add device 0000:00:01.0
(XEN) PCI add device 0000:00:02.0
(XEN) PCI add device 0000:00:14.0
(XEN) PCI add device 0000:00:16.0
(XEN) PCI add device 0000:00:16.3
(XEN) PCI add device 0000:00:19.0
(XEN) PCI add device 0000:00:1a.0
(XEN) PCI add device 0000:00:1b.0
(XEN) PCI add device 0000:00:1c.0
(XEN) PCI add device 0000:00:1c.1
(XEN) PCI add device 0000:00:1c.2
(XEN) PCI add device 0000:00:1c.4
(XEN) PCI add device 0000:00:1d.0
(XEN) PCI add device 0000:00:1f.0
(XEN) PCI add device 0000:00:1f.2
(XEN) PCI add device 0000:00:1f.3
(XEN) PCI add device 0000:01:00.0
(XEN) PCI phantom 0000:01:00.4
(XEN) PCI add device 0000:02:00.0
(XEN) PCI add device 0000:03:00.0
(XEN) PCI add device 0000:04:00.0
(XEN) PCI add device 0000:05:00.0
(XEN) PCI add device 0000:06:00.0
root#debian-guest-01:~# cat /proc/iomem
00000000-00000fff : reserved
00001000-0009ffff : System RAM
000a0000-000fffff : reserved
000f0000-000fffff : System ROM
00100000-3fffffff : System RAM
01000000-015bfd4c : Kernel code
015bfd4d-01b127ff : Kernel data
01c65000-01d3dfff : Kernel bss
f7b00000-f7b0ffff : 0000:00:00.0
f7b00000-f7b0ffff : ath9k
fee00000-fee00fff : Local APIC
root#debian-guest-01:~# lspci -vvvxxx
00:00.0 Network controller: Qualcomm Atheros AR928X Wireless Network Adapter (PCI-Express) (rev 01)
Subsystem: Qualcomm Atheros AR928X Wireless Network Adapter (PCI-Express)
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 25
Region 0: Memory at f7b00000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit-
Address: 00000000 Data: 0000
Capabilities: [60] Express (v1) Legacy Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L0s <512ns, L1 <64us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
Capabilities: [90] MSI-X: Enable- Count=1 Masked-
Vector table: BAR=0 offset=00000000
PBA: BAR=0 offset=00000000
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [140 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01
Status: NegoPending- InProgress-
Capabilities: [160 v1] Device Serial Number 00-00-00-00-00-00-00-00
Kernel driver in use: ath9k
Kernel modules: ath9k

Problems with changing productID on external device, failling to read correctly on Mac OS 10.9.4

I am having a slightly weird problem relating to reading a USB device on the Mac. The device is a Amplifier so it passes audio and HID messages.
OSX 10.9.4
Using IOKit USB Debug logging library.
When plugging in and attempting to setup the USB device for reading and writing, it fails when opening the Interface with error 0xE00002D5.
I should say that this device has worked correctly on the Mac with a different ProductID. Changing the ID to any other value apart from it's original value causes the above error.
Prior to this issue, the USB has worked correctly with three other devices for the past few years. If I change the ProductID back to one of the released values this works fine.
I am just a little confused about this sudden change of behavior. The USB device works well on the Windows platform.
I have traced the USB via logging enabled libraries and USB Prober, I'll include the failed log output (I'll apologize in advance for placing in question) I have removed the name of the amplifier and also the vendor and product id:
Feb 11 15:42:14.273 [6] Finding device driver for [REMOVED], matching user client dictionary, score: 89000
Feb 11 15:42:14.273 [6] Matched: idVendor ([REMOVED]) idProduct ([REMOVED])
Feb 11 15:42:14.273 [6] Didn't Match: nothing
Feb 11 15:42:14.279 [5] IOUSBDeviceUserClientV2[0xffffff80483a4c00]::initWithTask Owning PID is 1009, name is TestUSB
Feb 11 15:42:14.279 [6] +IOUSBDeviceUserClientV2[0xffffff80483a4c00]::start(0xffffff8048384b00)
Feb 11 15:42:14.279 [5] CompositeDriverInterestHandler received kIOMessageServiceIsAttemptingOpen with argument: 0
Feb 11 15:42:14.279 [6] [REMOVED][0xffffff8048384b00]::handleOpen - IOUSBDeviceUserClientV2[0xffffff80483a4c00] added to open set, _OPEN_CLIENTS count: 11
Feb 11 15:42:14.279 [6] -IOUSBDeviceUserClientV2[0xffffff80483a4c00]::start
Feb 11 15:42:14.279 [6] +IOUSBInterfaceUserClientV3[0xffffff80483a5000]::initWithTask
Feb 11 15:42:14.279 [5] IOUSBInterfaceUserClientV2[0xffffff80483a5000]::initWithTask Owning PID is 1009, name is TestUSB
Feb 11 15:42:14.279 [6] +IOUSBInterfaceUserClientV2[0xffffff80483a5000]::start(0xffffff803760f400)
Feb 11 15:42:14.279 [6] IOUSBInterface[0xffffff803760f400]::open calling super::open with gate
Feb 11 15:42:14.279 [6] IOUSBHIDDriver[0xffffff8037613400]::message type: 0xe0000101, argument = 0 isInactive = 0
Feb 11 15:42:14.279 [6] +IOUSBInterface[0xffffff803760f400]::handleOpen (device [REMOVED])
Feb 11 15:42:14.279 [6] IOUSBInterface[0xffffff803760f400]::handleOpen - IOUSBInterfaceUserClientV3[0xffffff80483a5000] added to open set
Feb 11 15:42:14.279 [6] -IOUSBInterfaceUserClientV2[0xffffff80483a5000]::start
Feb 11 15:42:14.279 [5] +IOUSBInterfaceUserClientV2[0xffffff80483a5000]::open isInactive: 0
Feb 11 15:42:14.279 [6] IOUSBInterface[0xffffff803760f400]::open calling super::open with gate
Feb 11 15:42:14.279 [6] IOUSBHIDDriver[0xffffff8037613400]::message type: 0xe0000101, argument = 0x20000 isInactive = 0
Feb 11 15:42:14.279 [6] +IOUSBInterface[0xffffff803760f400]::handleOpen (device [REMOVED])
Feb 11 15:42:14.279 [5] IOUSBInterface[0xffffff803760f400]::handleOpen - [0xffffff80483a5000] is a IOUSBInterfaceUserClientV3, exclusiveOpen = TRUE
Feb 11 15:42:14.279 [6] IOUSBInterface[0xffffff803760f400]::handleOpen - super::handleOpen returned 0x0
Feb 11 15:42:14.279 [2] IOUSBInterface[0xffffff803760f400]::open super::open failed (0xe00002be)
Feb 11 15:42:14.279 [2] IOUSBInterface[0xffffff803760f400]::open super::open failed (0x0)
Feb 11 15:42:14.279 [5] IOUSBInterfaceUserClientV2[0xffffff80483a5000]::open fOwner->open() failed. Returning kIOReturnExclusiveAccess
Feb 11 15:42:14.279 [5] -IOUSBInterfaceUserClientV2[0xffffff80483a5000]::open - returning 0xe00002c5 (device is exclusive access)
Feb 11 15:42:14.279 [6] IOUSBInterfaceUserClientV2[0xffffff80483a5000]::clientClose
Feb 11 15:42:14.279 [6] IOUSBInterfaceUserClientV2[0xffffff80483a5000]::ClientCloseEntry calling ClientCloseGated
Feb 11 15:42:14.279 [5] +IOUSBInterfaceUserClientV3[0xffffff80483a5000]::ClientCloseGated FNOTIFICATION_SET [0]
Feb 11 15:42:14.279 [5] +IOUSBInterfaceUserClientV2[0xffffff80483a5000]::ClientCloseGated isInactive = 0, fOutstandingIO = 0, FOWNER_WAS_RELEASED: 0
Feb 11 15:42:14.279 [6] IOUSBInterfaceUserClientV2[0xffffff80483a5000]::ClientCloseGated calling fOwner(0xffffff803760f400)->release()
Feb 11 15:42:14.279 [5] +IOUSBInterfaceUserClientV2[0xffffff80483a5000]::ClientCloseGated calling terminate()
Feb 11 15:42:14.279 [6] IOUSBInterfaceUserClientV3[0xffffff80483a5000]::terminate calling super::terminate
Feb 11 15:42:14.280 [6] -IOUSBInterfaceUserClientV3[0xffffff80483a5000]::terminate
Feb 11 15:42:14.280 [6] -IOUSBInterfaceUserClientV2[0xffffff80483a5000]::ClientCloseGated isInactive(1), fOutstandingIO (0)
Feb 11 15:42:14.280 [6] +IOUSBInterfaceUserClientV2[0xffffff80483a5000]::finalize(00000005), isInactive = 1, fOutstandingIO = 0, inGate: Yes
Feb 11 15:42:14.280 [6] +IOUSBInterfaceUserClientV2[0xffffff80483a5000]::stop(0xffffff803760f400), IsInactive: 1, outstandingIO: 0, inGate: Yes
Feb 11 15:42:14.280 [6] -IOUSBInterfaceUserClientV2[0xffffff80483a5000]::stop(0xffffff803760f400)
Feb 11 15:42:14.280 [6] IOUSBInterface[0xffffff803760f400]::handleIsOpen - IOUSBInterfaceUserClientV3[0xffffff80483a5000] has us open
Feb 11 15:42:14.280 [6] IOUSBInterface[0xffffff803760f400]::close called from an IOUSBInterfaceUserClientV2/V3 client that was not open exclusively, so no need to abort pipes
Feb 11 15:42:14.280 [6] IOUSBInterface[0xffffff803760f400]::close calling super::close with gate
Feb 11 15:42:14.280 [6] IOUSBInterface[0xffffff803760f400]::handleIsOpen - IOUSBInterfaceUserClientV3[0xffffff80483a5000] has us open
Feb 11 15:42:14.280 [6] +IOUSBInterface[0xffffff803760f400]::handleClose
Feb 11 15:42:14.280 [6] IOUSBInterface[0xffffff803760f400]::handleClose - IOUSBInterfaceUserClientV3[0xffffff80483a5000] removed from open set
Feb 11 15:42:14.280 [6] -IOUSBInterface[0xffffff803760f400]::handleClose
Feb 11 15:42:14.280 [6] IOUSBHIDDriver[0xffffff8037613400]::message type: 0xe0000110, argument = 0 isInactive = 0
Feb 11 15:42:14.280 [5] +IOUSBInterfaceUserClientV3[0xffffff80483a5000]::free
Feb 11 15:42:14.280 [6] +IOUSBInterfaceUserClientV2[0xffffff80483a5000]::free
Feb 11 15:42:14.280 [5] -IOUSBInterfaceUserClientV3[0xffffff80483a5000]::free
You may have guessed that I'm not a Mac expert, so any assistance would be greatly received.
Thank you.
Missing kext for the USB Device.
Many thanks for all that have viewed this.
Thanks to pmdj, I found out that a new kext needed to be created and loaded on to the system. All functionality is now working between the device and our application.
I will now add this addition to the installer.

Resources