I am new to qemu and am trying to learn kernel programming, I create an initrd which has busy box, but when I add a big tarbal ~80Mb in the cpio qemu fails to load.
I wanted to include golang in intrd, so that i could test the new kernel.
This is what is happening:
mfrw#kp ~/os/busybox/test_build
% ls
bin linuxrc sbin usr
mfrw#kp ~/os/busybox/test_build
% !find
find . | cpio -o -H newc | gzip > rootfs_bb.gz
cpio: File ./rootfs_bb.gz grew, 1261568 new bytes not copied
7374 blocks
mfrw#kp ~/os/busybox/test_build
% ls -ltrh
total 2.6M
drwxr-xr-x 2 mfrw mfrw 4.0K Mar 18 01:56 bin
lrwxrwxrwx 1 mfrw mfrw 11 Mar 18 01:56 linuxrc -> bin/busybox
drwxr-xr-x 2 mfrw mfrw 4.0K Mar 18 01:56 sbin
drwxr-xr-x 4 mfrw mfrw 4.0K Mar 18 15:24 usr
-rw-r--r-- 1 mfrw mfrw 2.6M Mar 18 15:31 rootfs_bb.gz
mfrw#kp ~/os/busybox/test_build
%
Then I run it using qemu with a freshly made kernel with rootfs = 2.6 M
mfrw#kp ~/os/linux_staging % qemu-system-x86_64 -nographic -no-reboot -kernel arch/x86/boot/bzImage -initrd ./../busybox/test_build/rootfs_bb.gz -append "panic=1 console=ttyS0 rdinit=/bin/sh"
[ 0.000000] Linux version 4.11.0-rc2+ (mfrw#kp) (gcc version 6.3.1 20170109 (GCC) ) #7 SMP Sat Mar 18 02:34:27 IST 2017
[ 0.000000] Command line: panic=1 console=ttyS0 rdinit=/bin/sh
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
...... the kernel boots fine
But When I include the tar.gz for golang in the rootfs, it shoots upto 80M and then fails to boot
mfrw#kp ~/os/busybox/test_build
% cp ~/go/go1.6.linux-amd64.tar.gz usr
mfrw#kp ~/os/busybox/test_build
% !fin
find . | cpio -o -H newc | gzip > rootfs_bb.gz
170406 blocks
mfrw#kp ~/os/busybox/test_build
% ls -ltrh
total 82M
drwxr-xr-x 2 mfrw mfrw 4.0K Mar 18 01:56 bin
lrwxrwxrwx 1 mfrw mfrw 11 Mar 18 01:56 linuxrc -> bin/busybox
drwxr-xr-x 2 mfrw mfrw 4.0K Mar 18 01:56 sbin
drwxr-xr-x 4 mfrw mfrw 4.0K Mar 18 15:34 usr
-rw-r--r-- 1 mfrw mfrw 82M Mar 18 15:34 rootfs_bb.gz
mfrw#kp ~/os/busybox/test_build
%
I try to run it with the same command and it fails to run...
mfrw#kp ~/os/linux_staging % qemu-system-x86_64 -nographic -no-reboot -kernel arch/x86/boot/bzImage -initrd ./../busybox/test_build/rootfs_bb.gz -append "panic=1 console=ttyS0 rdinit=/bin/sh"
.... no .. output
What am I doing wrong ? Any pointers please :)
I found the solution finally, did not give it enough memory so it was failing.
I finally ran it with
-m size=512
And it ran well
Related
I have installed python3.6 in my desktop which already had python3.5 and python2.7 inside. I change the default path for both python and python3 to python3.6 but it seems like python-config is still on python2.7 and python3-config is still on python3.5. How can I change the default python3-config or python-config to python3.6-config?
Thanks for the help!
so to elaborate on my comment, python comes usually with a version attached like so
ls -lisa /usr/local/bin/ | grep python
34138948 16 -r-xr-xr-x 2 root wheel 5248 15 Apr. 03:12 python3.7
34138954 0 lrwxr-xr-x 1 root wheel 17 15 Apr. 03:12 python3.7-config -> python3.7m-config
34138948 16 -r-xr-xr-x 2 root wheel 5248 15 Apr. 03:12 python3.7m
34138949 8 -r-xr-xr-x 1 root wheel 2936 15 Apr. 03:12 python3.7m-config
34127071 16 -r-xr-xr-x 1 root wheel 5216 15 Apr. 10:59 python3.9
34127072 8 -r-xr-xr-x 1 root wheel 3153 15 Apr. 10:59 python3.9-config
as you can see some names are already symlinks (the once with the -> ) so if you wanna access python3.9-config as python3-config you need to make a symlink with the name python3-config that points to the binary python3.9-config.
% sudo ln -s /usr/local/bin/python3.9-config /usr/local/bin/python3-config
so which should now give you
% which python3-config
/usr/local/bin/python3-config
looking at the directory again shows the symlink now
% ls -lisa /usr/local/bin | grep python3
34127092 0 lrwxr-xr-x 1 root wheel 31 22 Juni 14:09 python3-config -> /usr/local/bin/python3.9-config
34138948 16 -r-xr-xr-x 2 root wheel 5248 15 Apr. 03:12 python3.7
34138954 0 lrwxr-xr-x 1 root wheel 17 15 Apr. 03:12 python3.7-config -> python3.7m-config
34138948 16 -r-xr-xr-x 2 root wheel 5248 15 Apr. 03:12 python3.7m
34138949 8 -r-xr-xr-x 1 root wheel 2936 15 Apr. 03:12 python3.7m-config
34127071 16 -r-xr-xr-x 1 root wheel 5216 15 Apr. 10:59 python3.9
34127072 8 -r-xr-xr-x 1 root wheel 3153 15 Apr. 10:59 python3.9-config
cheers
Markus
When trying to enter into Docker VM in mac, I am getting screen is terminating error whereby I cannot go into Docker Desktop. In Mac, Docker host is not the mac as it runs as a VM. Any solutions?
Thank you for the help.
Workarounds
I think it's a bug since version 2.3.0.4 of Docker Desktop for Mac, as I have used screen successfully to access the Docker VM in the past on Mac without this issue. (edit: this issue appears to be still present at version 2.4.0.0)
There are some alternative methods to access the Docker VM. Here is one that worked for me:
workaround method 1
run this command to enter shell of the Docker VM:
docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -n -i sh
Then you can do what you need to do at the usual location for Docker volumes:
/ # ls -lah /var/lib/docker/volumes/
total 72
drwx------ 10 root root 4.0K Aug 27 11:34 .
drwx--x--x 15 root root 4.0K Oct 31 00:51 ..
drwxr-xr-x 3 root root 4.0K Aug 26 10:44 14ce94e174839f1947efa6fcbf5ac1fb2ea3b0f0b3f25311fee333ee374576b6
drwxr-xr-x 3 root root 4.0K Aug 25 12:34 34eb2bd80931ff954e1da80c5383beb4def61129d4005432b77080531cd10a5a
drwxr-xr-x 3 root root 4.0K Aug 27 11:34 830de2ce31519c921b50c448964b54517ca4461d337b56a9fd6e5b354ace3247
drwxr-xr-x 3 root root 4.0K Aug 25 12:34 data-layer_mongo_data
drwxr-xr-x 3 root root 4.0K Aug 26 11:01 ebb440184703bfad17184bd5ff74b677b50d74b8f0fbdd116506a1fcacfb00cd
drwxr-xr-x 3 root root 4.0K Aug 27 11:27 hawakening-services_back-end_log
drwxr-xr-x 3 root root 4.0K Aug 27 11:27 hawakening-services_front-end_log
drwxr-xr-x 3 root root 4.0K Aug 27 11:27 hawakening-services_mongo_data
-rw------- 1 root root 64.0K Oct 31 00:51 metadata.db
workaround method 2
Here's another one that also worked for me.
Run this command to enter shell of the Docker VM:
docker run --rm -it --privileged --pid=host walkerlee/nsenter -t 1 -m -u -i -n sh
As before, here is where the Docker volumes' directories are:
/ # ls -lah /var/lib/docker/volumes/
total 72
drwx------ 10 root root 4.0K Aug 27 11:34 .
drwx--x--x 15 root root 4.0K Oct 31 00:51 ..
drwxr-xr-x 3 root root 4.0K Aug 26 10:44 14ce94e174839f1947efa6fcbf5ac1fb2ea3b0f0b3f25311fee333ee374576b6
drwxr-xr-x 3 root root 4.0K Aug 25 12:34 34eb2bd80931ff954e1da80c5383beb4def61129d4005432b77080531cd10a5a
drwxr-xr-x 3 root root 4.0K Aug 27 11:34 830de2ce31519c921b50c448964b54517ca4461d337b56a9fd6e5b354ace3247
drwxr-xr-x 3 root root 4.0K Aug 25 12:34 data-layer_mongo_data
drwxr-xr-x 3 root root 4.0K Aug 26 11:01 ebb440184703bfad17184bd5ff74b677b50d74b8f0fbdd116506a1fcacfb00cd
drwxr-xr-x 3 root root 4.0K Aug 27 11:27 hawakening-services_back-end_log
drwxr-xr-x 3 root root 4.0K Aug 27 11:27 hawakening-services_front-end_log
drwxr-xr-x 3 root root 4.0K Aug 27 11:27 hawakening-services_mongo_data
-rw------- 1 root root 64.0K Oct 31 00:51 metadata.db
I'm writing a shell script that uses the shasum to check if the contents of a directory have changed.
On Linux and FreeBSD, the shasum have the same behavior when I do shasum <directory> however, on MacOS the shasum give me hashes for files only.
FreeBSD
$ shasum CONTENTS/
7f986e5e5289c59db1bba48df92ffe4707830aaa CONTENTS/
Linux
$ shasum CONTENTS/
7f986e5e5289c59db1bba48df92ffe4707830aaa CONTENTS/
MacOS
$ shasum CONTENTS/
shasum: CONTENTS/:
How could I calculate the hash of a directory in MacOS?
TRY 1: Using TAR with pipes
Tried to use but seems that this tar option doesn't work on MacOS.
tar cO CONTENTS/ | shasum
tar: Option -O is not permitted in mode -c
da39a3ee5e6b4b0d3255bfef95601890afd80709 -
TRY 2: Using FIND/EXEC
It was consistent between MacOS and FreeBSD, but Linux returned a weird hash
find CONTENTS -type f -exec shasum {} \; | sort -k 2 | shasum
Linux
c2ddb9bc5f543e956f5cdcc76750cb78cc5f26f3
FreeBSD
3ac2a9d4e2fc5d2d2ec3c7f612e680990cc35824
MacOS
3ac2a9d4e2fc5d2d2ec3c7f612e680990cc35824
OTHER FINDINGS ON TAR
tar would be excellent as it "archives" a folder and then I could shasum it, however the order of how tar "walk" the folder structure is not consistent across operating systems. As some helpers mentioned in the comments that I should use the same version of tar in all systems.
Just an example, on system 1 I have this order:
drwxr-xr-x 0 root wheel 0 27 Jul 07:23 usr/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f1/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f1/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f1/f0/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f1/f0/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f2/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f2/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f2/f1/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f2/f1/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f2/f1/f0/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f2/f1/f0/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f3/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f3/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f3/f2/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f3/f2/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f3/f2/f1/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f3/f2/f1/aaa
and on system 2 I have the following order:
drwxr-xr-x 0 root wheel 0 27 Jul 07:23 usr/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f1/
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f2/
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f3/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f3/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f3/f2/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f3/f2/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f3/f2/f1/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f3/f2/f1/aaa
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f2/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f2/f1/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f2/f1/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f2/f1/f0/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f2/f1/f0/aaa
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f1/aaa
drwxr-xr-x 0 root wheel 0 27 Jul 07:25 usr/f1/f0/
-rw-r--r-- 0 root wheel 0 27 Jul 07:25 usr/f1/f0/aaa
From a tar standpoint it if all good, but due to the order, the shasum produces a different hash.
CONCLUSION
shasum is consistent among Linux and BSDs to check an individual file hash, but, when it comes to directories the consistency happens only on MacOS and FreeBSD, perhaps due to how files are sorted.
If sorting is enforced using the find command, consistency is only obtained in FreeBSD and MacOS, however this method is time prohibitive as it takes a significant amount of time to calculate the hashes for every single file and then the whole structure hash.
Using tar to create a temporary file and then doing a shasum also found to be inconsistent between Linux and BSDs, perhaps because of difference in the archiving method.
I think the only way forward is to redesign my solution.
I had this problem some time ago and I ended up tarring the directory and generating a hash from the tar.
$ mkdir -p test
$ echo 1 > test/tmp
$ tar cO test/ | md5sum
7b18a99a8ccfef1ebbfd1e7a8b2852ee
$ echo 2 > test/tmp
$ tar cO test/ | md5sum
644042dd530157e604641ea89b4e9152
Note that if you write the same content to a file, the modified stat is updated and a new hash will be generated for that directory.
$ echo 2 > test/tmp
$ tar cO test/ | md5sum
da25819594f123563a837d5786e51950 -
$ echo 2 > test/tmp
$ tar cO test/ | md5sum
9407b64d43b809a5828a9fc2297b4e9c -
p.s. you should change md5sum for shasum :-)
EDIT:
Discussed it shortly with a friend who uses MacOSX and he came up with:
$ mkdir tmp; echo 1 > test/a;
$ tar -cf - tmp/ | md5sum
26c43adc9eca9f63279d08a0d145dd7d -
$ echo 2 > test/a
$ tar -cf - tmp/ | md5sum
dc3a68cac0b0224be9b202d86e69c5bd -
Note that MACosx uses md5 instead of md5sum.
I use usually use tmutil to control Timemachine instead of the GUI, which has some deficiencies to make it appealable to the masses. I tend to have a lot drives attached to my iMac, at this moment I have 11 with twenty total mounted partitions, but the drives and paritions change as I work on/with them. Sometimes many at a time.
I wrote a one-liner with the goal to report on the exclusion status of all drives in the /Volumes/ directory and an alias to it for simple reuse. alias tmutilvol="find /Volumes/ -maxdepth 1 -exec tmutil isexcluded {} \;" It works, well mostly. The typical output looks this is,
[Included] /Volumes
[Excluded] /Volumes/Backup-Lion
[Excluded] /Volumes/Backups
[Excluded] /Volumes/chris
[Excluded] /Volumes/El Capitan Temp Holding
[Excluded] /Volumes/Font
[Excluded] /Volumes/Hades
[Excluded] /Volumes/iMac Bad
[Excluded] /Volumes/iMac Clone
[Excluded] /Volumes/iMac Clone New
[Excluded] /Volumes/iMac ElCap
[Excluded] /Volumes/iMac HD
[Excluded] /Volumes/iMac HD New
[Excluded] /Volumes/iMac HS Clone
[Excluded] /Volumes/Macintosh HD Clone 1
[Excluded] /Volumes/maclaptop
[Excluded] /Volumes/Old Laptop
[Excluded] /Volumes/Old-Timemachine-Garnet
[Excluded] /Volumes/Old-Timemachine-Lion
[Excluded] /Volumes/TEMPORARY
[Excluded] /Volumes/Time Machine Mirror
The problem is /Volumes/iMac HD. This is not the actual drive, it is a symlink to it.
drwxr-xr-x 33 chris staff 1190 Jan 17 12:31 Backup-Lion
drwxrwxr-x 20 chris staff 748 Nov 11 22:03 Backups
drwxrwxr-x 107 chris staff 3706 Nov 13 16:26 El Capitan Temp Holding
drwxrwxr-x 82 chris staff 2856 Jan 19 16:07 Font
drwxr-xr-x 260 chris staff 8908 Jan 22 11:51 Hades
drwxr-xr-x# 41 chris staff 1462 Nov 11 22:03 Macintosh HD Clone 1
drwxr-xr-x 24 chris staff 884 Nov 11 22:03 Old Laptop
drwxr-xr-x 6 chris staff 272 Jan 17 12:31 Old-Timemachine-Garnet
drwxrwxr-x# 20 chris staff 748 Jan 19 14:49 Old-Timemachine-Lion
drwxr-xr-x 20 chris staff 748 Jan 22 11:54 TEMPORARY
drwxr-xr-x 12 chris staff 476 Jan 23 01:30 Time Machine Mirror
drwx------# 6 chris staff 272 Jan 16 16:18 chris
drwxrwxrwx 36 chris staff 1292 Jan 19 00:15 iMac Bad
drwxrwxrwx 29 chris staff 1054 Nov 13 16:26 iMac Clone
drwxr-xr-x 6 chris staff 272 Jan 17 16:05 iMac Clone New
drwxr-xr-x 30 chris staff 1088 Jan 16 17:31 iMac ElCap
lrwxr-xr-x 1 root wheel 1 Jan 16 20:34 iMac HD -> /
drwxr-xr-x 6 chris staff 272 Jan 17 16:05 iMac HD New
drwxrwxr-x 26 root wheel 952 Nov 13 16:26 iMac HS Clone
drwxrwxrwx 40 chris staff 1428 Jan 19 15:40 maclaptop
The result of tmutil isexcluded is actually for the symlink and not "/"
Kaze:~ chris$ tmutil isexcluded /
[Included] /
I want to know if there is a way to get find to follow and pass in the destination of the symlink. I read the find man page and about the -L option and thought that would do it and it does not. Some digging on stackexchange https://unix.stackexchange.com/questions/31114/find-usage-with-l explained why it doesn't work. What I have not been able to find is any clue as to how I might do what want and keep a simple one-liner or if I am going to have to write a function for this and deal with symlinks in some convoluted way.
I am hoping that someone with a lot more knowledge of CLI and bash than I might know how to do what I want simply.
Have you tried?
readlink "/Volumes/iMac HD"
one liner for mac:
find /Volumes -maxdepth 1 -exec bash -c 'N="{}"; L=`readlink "{}"`; tmutil isexcluded "${L:-$N}"' \;
Explanation:
have to use two variables, $L for readlink output, $N for original name.
${L:-$N} basically just output $L if its not empty, else output $N
Note that I have to prefix $L with "/Volumes/" as MacOS's readlink does not output the absolute path. Linux "readlink -f" solves this issue.
I install lua success in my mac, but i got "-bash: lua: command not found" message when i run lua cmd, but i run /usr/loca/bin/lua success, i confirm the path that lua installed is included in PATH. I record some information as follow:
lxr:bin wang$ pwd
/usr/local/bin
lxr:bin wang$ echo $PATH
.;/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:JAVA_HOME/bin
lxr:bin wang$ lua
-bash: lua: command not found
lxr:bin wang$ pwd
/usr/local/bin
lxr:bin wang$ ls -l
total 872
-rwxr-xr-x 1 root wheel 80 11 23 00:40 VBoxAutostart
-rwxr-xr-x 1 root wheel 82 11 23 00:40 VBoxBalloonCtrl
-rwxr-xr-x 1 root wheel 80 11 23 00:40 VBoxBugReport
-rwxr-xr-x 1 root wheel 77 11 23 00:40 VBoxDTrace
-rwxr-xr-x 1 root wheel 79 11 23 00:40 VBoxHeadless
-rwxr-xr-x 1 root wheel 77 11 23 00:40 VBoxManage
-rwxr-xr-x 1 root wheel 79 11 23 00:40 VBoxVRDP
-rwxr-xr-x 1 root wheel 77 11 23 00:40 VirtualBox
lrwxr-xr-x 1 root wheel 14 3 28 2016 git -> ../git/bin/git
lrwxr-xr-x 1 root wheel 37 3 28 2016 git-credential-osxkeychain -> ../git/bin/git-credential-osxkeychain
lrwxr-xr-x 1 root wheel 24 3 28 2016 git-cvsserver -> ../git/bin/git-cvsserver
lrwxr-xr-x 1 root wheel 20 3 28 2016 git-shell -> ../git/bin/git-shell
lrwxr-xr-x 1 root wheel 26 3 28 2016 git-upload-pack -> ../git/bin/git-upload-pack
lrwxr-xr-x 1 root wheel 15 3 28 2016 gitk -> ../git/bin/gitk
-rwxr-xr-x 1 root wheel 221128 12 1 15:53 lua
-rwxr-xr-x 1 root wheel 150520 12 1 15:53 luac
-rwxr-xr-x 1 root wheel 2670 11 28 2014 pstorm
-rwxr-xr-x 1 root wheel 1394 3 8 2017 ssh-copy-id
-rwxr-xr-x 1 root wheel 75 11 23 00:40 vbox-img
-rwxr-xr-x 1 root wheel 77 11 23 00:40 vboxwebsrv
lxr:bin wang$ ./lua
Lua 5.3.0 Copyright (C) 1994-2015 Lua.org, PUC-Rio
>
lxr:bin wang$
The first two characters in $PATH should probably be .:, not .;.