I tried the "DC CIVAC ...." instruction in Arm-V8 ISA to clear a cache line for apple M1. But seems it didn't work for flushing one cache line. Did anyone successfully flush the cache line on M1?
Related
Stuck at "Allocated cache and file handles"
I download SnapShot and unzipped
replaced files(chaindata and tricache)
Followed this instruction:
https://github.com/charliedimaggio/bsc-fullnode-installer/blob/main/snapshot-sync.md
SnapShot:
https://tf-dex-prod-public-snapshot-site1.s3-accelerate.amazonaws.com/geth-20220327-prune-ancient.tar.lz4?AWSAccessKeyId=AKIAYINE6SBQPUZDDRRO&Signature=Kw3LcdoYQ81pf3jvj0icLQCjkSE%3D&Expires=1651046857
When starting the project, same thing Stuck at "Allocated cache and file handles"
I am trying to test a IO kit sample app that implements a filter scheme. See section 10.7.10 here for context. As the source there is hard to cut and paste (and when I tried I got errors), I am using the source from here which builds.
In the last figure on that page (1038) it has a series of steps for creating and mounting a new HFS volume. The idea is to see that the KEXT is working and doing encryption. I have built the sample KEXT and tested with it, but I always get "No mountable file systems" error when I make the "open" call. I have verified the KEXT is properly loaded.
If I remove the "-partitionType" I am able to manipulate the volume normally (though this doesn't involve the KEXT).
Can someone please suggest what might be going wrong here?
Part of my confusion in figure 1038's script is the "detach" command that is run before the open. It fails for me, but I am not sure if that is what is causing the "open" after it to fail, or is unrelated.
$ sudo kextload /System/Library/Extensions/IOKitDriverTestApp.kext/
$ hdiutil create -size 32m -partitionType osxbook_HFS /tmp/crypto.dmg
created: /tmp/crypto.dmg
$ hdiutil attach -nomount /tmp/crypto.dmg
/dev/disk2 Apple_partition_scheme
/dev/disk2s1 Apple_partition_map
/dev/disk2s2 osxbook_HFS
$ diskutil list
/dev/disk0 (internal, physical):
[cut]
/dev/disk1 (synthesized):
[cut]
/dev/disk2 (disk image):
#: TYPE NAME SIZE IDENTIFIER
0: Apple_partition_scheme +33.6 MB disk2
1: Apple_partition_map 32.3 KB disk2s1
2: osxbook_HFS 33.5 MB disk2s2
$ newfs_hfs -v Crypto /dev/rdisk2s2
Initialized /dev/rdisk2s2 as a 32 MB case-insensitive HFS Plus volume
$ hdiutil detach /dev/disk2
hdiutil: detach failed - No such file or directory
$ open /tmp/crypto.dmg
==> Get popup window with error "crypto.dmg" "No mountable file systems"
UPDATE: the question originally targeted the case when the KEXT was not loaded. Since I got that to work I changed it to reflect the case with the KEXT, which is more important anyway.
UPDATE2: It seems that eventually the kext is getting unloaded (possibly crashing). Sometimes it seems around the time of the "open", sometimes after. I tried adding statements like IOLog("CSD: init"), but none of them show up in the system log or console log. Looking for how else I can triage this (without needing a separate machine).
UPDATE3: Below is the output of when I try to start the KEXT. There are a bunch of warnings but since I have the signing check disabled I think it is still loading. Also I see it loaded via kextstat.
The other strange thing is I don't see any signs of this KEXT in the output of "ioreg", nor do I see any of the print statements from IOLog().
My-MacBook-Pro:Extensions myuser$ sudo kextutil -v /System/Library/Extensions/IOKitDriverTestApp.kext/
Defaulting to kernel file '/System/Library/Kernels/kernel'
Kext with invalid signatured (-67050) allowed: <OSKext 0x7fa53703ec70 [0x7fff8fe6faf0]> { URL = "file:///System/Library/Extensions/IOKitDriverTestApp.kext/", ID = "com.osxbook.driver.IOKitDriverTestApp" }
Code Signing Failure: code signature is invalid
Warnings:
Personality CFBundleIdentifier differs from containing kext's (not necessarily a mistake, but rarely done):
SimpleCryptoDisk
Personality CFBundleIdentifier names a kext that can't be found:
'SimpleCryptoDisk' -> 'com.osxbook.driver.SimpleCryptoDisk'
Warnings:
Personality CFBundleIdentifier differs from containing kext's (not necessarily a mistake, but rarely done):
SimpleCryptoDisk
Personality CFBundleIdentifier names a kext that can't be found:
'SimpleCryptoDisk' -> 'com.osxbook.driver.SimpleCryptoDisk'
/System/Library/Extensions/IOKitDriverTestApp.kext appears to be loadable (not including linkage for on-disk libraries).
Loading /System/Library/Extensions/IOKitDriverTestApp.kext.
/System/Library/Extensions/IOKitDriverTestApp.kext successfully loaded (or already loaded).
Invalid signature -67050 for kext <OSKext 0x7fa53703ec70 [0x7fff8fe6faf0]> { URL = "file:///System/Library/Extensions/IOKitDriverTestApp.kext/", ID = "com.osxbook.driver.IOKitDriverTestApp" }
UPDATE4: I adjusted the bundle ID in the plist so it matches, and now it is not giving the "names a kext that can't be found" error:
My-MacBook-Pro:Debug myuser$ sudo kextutil -v /System/Library/Extensions/IOKitDriverTestApp.kext/
Password:
Defaulting to kernel file '/System/Library/Kernels/kernel'
Kext with invalid signatured (-67050) allowed: <OSKext 0x7f8be663ecf0 [0x7fff866bcaf0]> { URL = "file:///System/Library/Extensions/IOKitDriverTestApp.kext/", ID = "com.osxbook.driver.IOKitDriverTestApp" }
Code Signing Failure: code signature is invalid
/System/Library/Extensions/IOKitDriverTestApp.kext appears to be loadable (not including linkage for on-disk libraries).
Loading /System/Library/Extensions/IOKitDriverTestApp.kext.
/System/Library/Extensions/IOKitDriverTestApp.kext successfully loaded (or already loaded).
Invalid signature -67050 for kext <OSKext 0x7f8be663ecf0 [0x7fff866bcaf0]> { URL = "file:///System/Library/Extensions/IOKitDriverTestApp.kext/", ID = "com.osxbook.driver.IOKitDriverTestApp" }
UPDATE5:
Now my OS consistently crashes every time I load the KEXT and then execute "hdiutil attach -nomount /tmp/crypto.dmg" (after creating /tmp/crypto.dmg).
It seems clear my KEXT is running now, but for some reason I still don't see the IOLog() statements anywhere. Will have to look through the system logs and see if there is any indication why it is crashing.
UPDATE6: I have been able to get a symbolicated kernel panic, but it is pointing to a different KEXT (which I don't think I have the source too). Below is the relevant part of it.
I tried using another version of the same test KEXT (here) without any changes (I have removed the IOLog() statements for now since they weren't showing up).
I see a note this was tested on Mac OS 10.6, but has anyone gotten this sample to work on 10.13.16 (High Sierra)?
...
0xffffff800c94f540 : 0xffffff800cb8776f mach_kernel : _kernel_trap + 0x70f
0xffffff800c94f6b0 : 0xffffff800ca1e1e0 mach_kernel : _return_from_trap + 0xe0
0xffffff800c94f6d0 : 0xffffff7f8d527251 com.apple.iokit.IOStorageFamily : __ZN9IOStorage8completeEP19IOStorageCompletioniy + 0x27
0xffffff81f7a4b980 : 0xffffff7f8d51a0c4 com.apple.iokit.IOStorageFamily : __ZN20IOBlockStorageDriver24prepareRequestCompletionEPvS0_iy + 0xc2
0xffffff81f7a4b9e0 : 0xffffff7f903afa7a com.apple.driver.DiskImages : __ZN13IOHDIXCommand8completeEiy + 0x26
0xffffff81f7a4ba00 : 0xffffff7f903adbfe com.apple.driver.DiskImages : __ZN22IOHDIXHDDriveOutKernel12processReplyEPK13HDIReplyOOL64P18IOMemoryDescriptor + 0x2be
0xffffff81f7a4ba60 : 0xffffff7f903aee9b com.apple.driver.DiskImages : __ZN32IOHDIXHDDriveOutKernelUserClient14processReply64EPK13HDIReplyOOL64 + 0xd3
0xffffff81f7a4bac0 : 0xffffff800d0c3959 mach_kernel : _shim_io_connect_method_structureI_structureO + 0x1c9
0xffffff81f7a4bb20 : 0xffffff800d0c1ae0 mach_kernel : __ZN12IOUserClient14externalMethodEjP25IOExternalMethodArgumentsP24IOExternalMethodDispatchP8OSObjectPv + 0x340
0xffffff81f7a4bb70 : 0xffffff800d0ca567 mach_kernel : _is_io_connect_method + 0x217
...
UPDATE7: I gave up on trying to use the SimpleCryptoDisk.cpp test app and started from scratch, adding a little bit of code at a time. I've had some success (the driver starts and I can see logging), though I don't really have anything working yet.
If I get stuck on this I may post another question, but I'll go ahead and mark pmdj's answer as correct since he answered my original question.
If anyone knows of a modified version of SimpleCryptoDisk that works on High Sierra I'd still like to know, however.
To answer your original questions:
[Why "No mountable file systems" when not loading the kext and using partition type "osxbook_HFS"]
As you can see, the Apple Partition Scheme uses arbitrary text strings for identifying partition type. macOS uses this string verbatim as the "content hint" which selects which file system type to attempt to automount on that volume. The HFS+ file system is only looking for Apple_HFS (and Apple_HFSX for case-sensitive)
The idea behind the kext is that it itself matches the newly defined content hint and publishes a new storage nub object which in turn has the HFS+ content hint for the filesystem to pick up.
Why hdiutil detach was failing
$ hdiutil detach /dev/disk2
hdiutil: detach failed - No such file or directory
On current versions of macOS, you need to rewrite the command as either:
$ hdiutil detach disk2
or
$ hdiutil detach /Volumes/VOLUMENAME
I can't immediately answer your current question of why it's not working with the kext, but I can suggest you look at whether your kext is behaving as expected with regard to the IO Registry. Check with IORegistryExplorer or ioreg whether it's matching the volume with the special partition type, and whether it's creating and registering the virtual (decrypted) volume with the Apple_HFS content hint. Add the ioreg output (ioreg -w 0 -lirc YOUR_DRIVERS_CLASSNAME) to your question if unsure.
I noticed every time I launch a Playground in Xcode (version 9.3) a daemon starts in the background and uses more that 100% of my CPU!
I deleted Xcode's cache and its DerivedData folder but no change.
Does anyone know how to resolve this problem?
diagnosticd is a daemon you need to stop/kill with launchctl. The daemon you most likely need to stop has 'SimDevice' in its name. To get the exact name do:
/bin/launchctl list | grep SimDevice
Then with the appropriate name(s) (there could be more than 1) do:
/bin/launchctl stop 'name'
In my case 'name' was: com.apple.CoreSimulator.SimDevice.380EFF06-B636-49CD-851E-5DB4890641AE
If you use Playground a lot and you usually have more than 1 simulator eating your CPU run this script instead:
/bin/launchctl list | grep SimDevice | awk '{print $3}' | xargs -I %s /bin/launchctl stop %s
Oh, and dont forget to file a bug report with apple here: https://bugreport.apple.com/web/
Btw, its kinda fun to see playground/diagnosticd struggle with your code; if you create a bug or some other elaborate syntax, cpu goes up. If you clear it, cpu goes down (just a bit). Almost hilarious how Apple implemented this...
As a temporary workaround, I run this script after starting up a playground to kill the rogue diagnosticd and homed processes:
#!/bin/bash
kill $(ps -ef | grep Xcode.app | egrep "diagnosticd|homed" | awk '{ print $2 }')
Edit: This seems to be a problem with iOS playgrounds only, so another (less drastic) workaround is to use a macOS playgrounds for those cases where no UIKit components are needed.
2nd Edit: This bug appears to be fixed in Xcode 9.3.1 (available via MAS or direct download from developer.apple.com)
3rd Edit: 9.3.1 doesn't fully fix this, despite what the Release Notes say. According to the comments in my bug report, this is only be fully fixed in 9.4.
Get Xcode 9.4 Beta. It resolved my issue.
killing homed is not good solution and playground is not acting correctly anyway.
Had same issue and there was no way around it. People filed bugs as far back as April 8th.
Best solution is to download Xcode 9.4 Beta from Apple Developer Page.
This worked for me and CPU usage is low and Xcode doesn't crash on playgrounds anymore.
I also had the same issue with Xcode v 11.2.1(11B53)
It seems the issue happened after updating Mac OS and Xcode.
I found out that diagnosticd has something to do with Xcode simulators.
So I tried to Erase All Content and Settings... in Simulator menu.
Please refer to above picture.
After erasing all content and settings the issue was resolved. :D
This is a known bug, attributed by Apple to “over-enthusiastic logging by the MediaRemote subsystem”. There is no official workaround at this time.
See https://forums.developer.apple.com/message/305885
Same problem here, the only temporary solution I found was to kill 'homed', which was flooding the system with errors like
MediaRemote reply error: <error: 0x11162ed20> { count = 1, transaction: 0, voucher = 0x0, contents =
"XPCErrorDescription" => <string: 0x11162efc8> { length = 18, contents = "Connection invalid" }
}
I suppose diagnosticd, designed to report errors automatically, simply went in overdrive mode.
That's all I have for now...
eCryptfs is a POSIX-compliant encrypted filesystem that has been part of the mainline Linux Kernel since version 2.6.19.
When I try to insert the module (ecryptfs.ko), I get the following error:
insmod: error inserting 'ecryptfs.ko': -1 Cannot allocate memory
Can some one please help me out?
below is the dmesg
Failed to allocate one or more kmem_cache objects
kmem_cache_create: duplicate cache ecryptfs_auth_tok_list_item
Pid: 3332, comm: insmod Tainted: G O 3.2.2+ #1
Call Trace:
[<c102bfe0>] ? printk+0x15/0x17
[<c10878b6>] kmem_cache_create+0x41c/0x458
[<d0ebd038>] ecryptfs_init+0x38/0x1b1 [ecryptfs]
[<c1001071>] do_one_initcall+0x71/0x118
[<d0ebd000>] ? 0xd0ebcfff
[<c1055703>] sys_init_module+0x60/0x18c
[<c12db9b0>] sysenter_do_call+0x12/0x36
ecryptfs_init_kmem_caches: ecryptfs_auth_tok_list_item: kmem_cache_create
failed
Failed to allocate one or more kmem_cache objects
Start with the error you are seeing in dmesg:
kmem_cache_create: duplicate cache ecryptfs_auth_tok_list_item
When the ecryptfs module is loaded the first thing it does is create a bunch of memory caches for itself. The error suggests that there is already a cache with that name.
You can check if the cache already exists by looking at sysfs:
$ ls -ld /sys/kernel/slab/ecryptfs*
NB. It may not show up in /proc/slabinfo due to slab merging.
If you see any ecryptfs slabs that suggests the ecryptfs module is already loaded, or it is already built into your kernel.
Normally the module loader would not let you load the same module twice, but perhaps you have done something weird to confuse it.
A likely cause of something like this happening is if one recompiles and installs a kernel and its modules but forgets to mount /boot before installing the kernel. After a reboot, one will then run with the old kernel but new modules. In any event, check that the running kernel is current, and reinstall both the kernel and the modules if in doubt:
mount /boot
cd /usr/src/linux
make && make install && make modules_install
I have done the above steps and error was solved
I'm compiling a custom kernel under Ubuntu and I'm running into the problem that my kernel doesn't seem to know where to look for firmware. Under Ubuntu 8.04, firmware is tied to kernel version the same way driver modules are. For example, kernel 2.6.24-24-generic stores its kernel modules in:
/lib/modules/2.6.24-24-generic
and its firmware in:
/lib/firmware/2.6.24-24-generic
When I compile the 2.6.24-24-generic Ubuntu kernel according the "Alternate Build Method: The Old-Fashioned Debian Way" I get the appropriate modules directory and all my devices work except those requiring firmware such as my Intel wireless card (ipw2200 module).
The kernel log shows for example that when ipw2200 tries to load the firmware the kernel subsystem controlling the loading of firmware is unable to locate it:
ipw2200: Detected Intel PRO/Wireless 2200BG Network Connection
ipw2200: ipw2200-bss.fw request_firmware failed: Reason -2
errno-base.h defines this as:
#define ENOENT 2 /* No such file or directory */
(The function returning ENOENT puts a minus in front of it.)
I tried creating a symlink in /lib/firmware where my kernel's name pointed to the 2.6.24-24-generic directory, however this resulted in the same error. This firmware is non-GPL, provided by Intel and packed by Ubuntu. I don't believe it has any actual tie to a particular kernel version. cmp shows that the versions in the various directories are identical.
So how does the kernel know where to look for firmware?
Update
I found this solution to the exact problem I'm having, however it no longer works as Ubuntu has eliminated /etc/hotplug.d and no longer stores its firmware in /usr/lib/hotplug/firmware.
Update2
Some more research turned up some more answers. Up until version 92 of udev, the program firmware_helper was the way firmware got loaded. Starting with udev 93 this program was replaced with a script named firmware.sh providing identical functionality as far as I can tell. Both of these hardcode the firmware path to /lib/firmware. Ubuntu still seems to be using the /lib/udev/firmware_helper binary.
The name of the firmware file is passed to firmware_helper in the environment variable $FIRMWARE which is concatenated to the path /lib/firmware and used to load the firmware.
The actual request to load the firmware is made by the driver (ipw2200 in my case) via the system call:
request_firmware(..., "ipw2200-bss.fw", ...);
Now somewhere in between the driver calling request_firmware and firmware_helper looking at the $FIRMWARE environment variable, the kernel package name is getting prepended to the firmware name.
So who's doing it?
From the kernel's perspective, see /usr/src/linux/Documentation/firmware_class/README:
kernel(driver): calls request_firmware(&fw_entry, $FIRMWARE, device)
userspace:
- /sys/class/firmware/xxx/{loading,data} appear.
- hotplug gets called with a firmware identifier in $FIRMWARE
and the usual hotplug environment.
- hotplug: echo 1 > /sys/class/firmware/xxx/loading
kernel: Discard any previous partial load.
userspace:
- hotplug: cat appropriate_firmware_image > \
/sys/class/firmware/xxx/data
kernel: grows a buffer in PAGE_SIZE increments to hold the image as it
comes in.
userspace:
- hotplug: echo 0 > /sys/class/firmware/xxx/loading
kernel: request_firmware() returns and the driver has the firmware
image in fw_entry->{data,size}. If something went wrong
request_firmware() returns non-zero and fw_entry is set to
NULL.
kernel(driver): Driver code calls release_firmware(fw_entry) releasing
the firmware image and any related resource.
The kernel doesn't actually load any firmware at all. It simply informs userspace, "I want a firmware by the name of xxx", and waits for userspace to pipe the firmware image back to the kernel.
Now, on Ubuntu 8.04,
$ grep firmware /etc/udev/rules.d/80-program.rules
# Load firmware on demand
SUBSYSTEM=="firmware", ACTION=="add", RUN+="firmware_helper"
so as you've discovered, udev is configured to run firmware_helper when the kernel asks for firmware.
$ apt-get source udev
Reading package lists... Done
Building dependency tree
Reading state information... Done
Need to get 312kB of source archives.
Get:1 http://us.archive.ubuntu.com hardy-security/main udev 117-8ubuntu0.2 (dsc) [716B]
Get:2 http://us.archive.ubuntu.com hardy-security/main udev 117-8ubuntu0.2 (tar) [245kB]
Get:3 http://us.archive.ubuntu.com hardy-security/main udev 117-8ubuntu0.2 (diff) [65.7kB]
Fetched 312kB in 1s (223kB/s)
gpg: Signature made Tue 14 Apr 2009 05:31:34 PM EDT using DSA key ID 17063E6D
gpg: Can't check signature: public key not found
dpkg-source: extracting udev in udev-117
dpkg-source: unpacking udev_117.orig.tar.gz
dpkg-source: applying ./udev_117-8ubuntu0.2.diff.gz
$ cd udev-117/
$ cat debian/patches/80-extras-firmware.patch
If you read the source, you'll find that Ubuntu wrote a firmware_helper which is hard-coded to first look for /lib/modules/$(uname -r)/$FIRMWARE, then /lib/modules/$FIRMWARE, and no other locations. Translating it to sh, it does approximately this:
echo -n 1 > /sys/$DEVPATH/loading
cat /lib/firmware/$(uname -r)/$FIRMWARE > /sys/$DEVPATH/data \
|| cat /lib/firmware/$FIRMWARE > /sys/$DEVPATH/data
if [ $? = 0 ]; then
echo -n 1 > /sys/$DEVPATH/loading
echo -n -1 > /sys/$DEVPATH/loading
fi
which is exactly the format the kernel expects.
To make a long story short: Ubuntu's udev package has customizations that always look in /lib/firmware/$(uname -r) first. This policy is being handled in userspace.
Wow this is very useful information and it led me to the solution for my problem when making a custom USB kernel module for a device requiring firmware.
Basically, every Ubuntu brings a new rehash of hal,sysfs,devfs,udev,and so on...and things just change. In fact I read they stopped using hal.
So let's reverse engineer this yet again so it's pertinent to the latest [Ubuntu] systems.
On Ubuntu Lucid (the latest at time of writing), /lib/udev/rules.d/50-firmware.rules is used. This file calls the binary /lib/udev/firmware, where magic happens.
Listing: /lib/udev/rules.d/50-firmware.rules
# firmware-class requests, copies files into the kernel
SUBSYSTEM=="firmware", ACTION=="add", RUN+="firmware --firmware=$env{FIRMWARE} --devpath=$env{DEVPATH}"
The magic should be something along these lines (source: Linux Device Drivers, 3rd Ed., Ch. 14: The Linux Device Model):
echo 1 to loading
copy firmware to data
on failure, echo -1 to loading and halt firmware loading process
echo 0 to loading (signal the kernel)
then, a specific kernel module receives the data and pushes it to the device
If you look at Lucid's source page for udev, in udev-151/extras/firmware/firmware.c, the source for that firmware /lib/udev/firmware binary, that's exactly what goes on.
Excerpt: Lucid source, udev-151/extras/firmware/firmware.c
util_strscpyl(datapath, sizeof(datapath), udev_get_sys_path(udev), devpath, "/data", NULL);
if (!copy_firmware(udev, fwpath, datapath, statbuf.st_size)) {
err(udev, "error sending firmware '%s' to device\n", firmware);
set_loading(udev, loadpath, "-1");
rc = 4;
goto exit;
};
set_loading(udev, loadpath, "0");
Additionally, many devices use an Intel HEX format (textish files containing checksum and other stuff) (wiki it i have no reputation and no ability to link). The kernel program ihex2fw (called from Makefile in kernel_source/lib/firmware on .HEX files) converts these HEX files to an arbitrary-designed binary format that the Linux kernel then picks up with request_ihex_firmware, because they thought reading text files in the kernel was silly (it would slow things down).
On current Linux systems, this is handled via udev and the firmware.agent.
Linux 3.5.7 Gentoo, I have the same issue.
SOLVED:
emerge ipw2200-firmware
Then go to /usr/src/linux
make menucofig
on device driver, remove all wirless drivers don't needed, set Intell 2200 as module and recompile.
make
make modules_install
cp arch/x86/boot/bzImage /boot/kernel-yourdefault