I'm trying to write a script to extract the original download URL from disk images downloaded with Safari on OS X using xattr, so that I can rename them but still easily obtain their original names for reference.
This command prints the hex representation of the URL that the given file was downloaded from, as an example:
xattr -p com.apple.metadata:kMDItemWhereFroms *.dmg
gives
62 70 6C 69 73 74 30 30 A1 01 5F 10 4F 68 74 74
70 3A 2F 2F 61 64 63 64 6F 77 6E 6C 6F 61 64 2E
61 70 70 6C 65 2E 63 6F 6D 2F 4D 61 63 5F 4F 53
5F 58 2F 6D 61 63 5F 6F 73 5F 78 5F 31 30 2E 36
2E 31 5F 62 75 69 6C 64 5F 31 30 62 35 30 34 2F
30 34 31 35 30 37 33 61 2E 64 6D 67 08 0A 00 00
00 00 00 00 01 01 00 00 00 00 00 00 00 02 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 5C
The URL starts at the 14th byte (if I counted correctly) and is NULL terminated. How can I format this string so that I get a string output as follows:
http://adcdownload.apple.com/Mac_OS_X/mac_os_x_10.6.1_build_10b504/0415073a.dmg
(don't worry, this link doesn't work unless you're logged in to ADC)
...essentially, the same thing Finder will display in Get Info. I tried piping xattr's output to xxd but I'm not sure how to specify the offset so the string starts at the right place.
So, after looking at the binary data returned by xattr -p, I realized that it was actually a binary plist... hence "bplist" at the front of the data. For some reason I didn't notice this before, but in light of this, here's a proper solution that should work on every OS X from 10.5 to 10.8.
To avoid duplication, I'll link to the source instead of pasting it: https://github.com/jakepetroules/wherefrom
Related
I've got some code to create labels in Gmail, which usually works fine. But now the requirement is to create a label with Japanese characters, specifically "アーカイブ". I am encoding the json like this:
7B 0D 0A 22 6E 61 6D 65 22 3A 22 E3 82 A2 E3 83 {.."name":".....
BC E3 82 AB E3 82 A4 E3 83 96 22 2C 0D 0A 22 6D ..........",.."m
65 73 73 61 67 65 4C 69 73 74 56 69 73 69 62 69 essageListVisibi
6C 69 74 79 22 3A 22 73 68 6F 77 22 2C 0D 0A 22 lity":"show",.."
6C 61 62 65 6C 4C 69 73 74 56 69 73 69 62 69 6C labelListVisibil
69 74 79 22 3A 22 6C 61 62 65 6C 53 68 6F 77 22 ity":"labelShow"
0D 0A 7D 0D 0A 00 00 00 00 00 00 00 00 00 00 00 ..}.............
As you can see, the first character is the UTF8 sequence E3 82 A2, which if you look at this table (https://www.utf8-chartable.de/unicode-utf8-table.pl?start=12352&names=-) seems to be correct for that first character. The others look OK also.
As a test, I created a Japanese folder with that name in the UI, then got a dump of the json that Gmail produces when I get a list of existing folders. What Gmail produces is exactly the same as what I'm trying to import. So I don't see what I could be doing wrong here. Any help appreciated.
Never mind this - turns out my Japanese characters translate to "Archive" which is apparently a reserved folder name.
Application is reading the message from MQ but showing null. Queue is empty after starting the application. When I stop the application it is going back to the queue and showing as an uncommitted message
protected final XaTransactedJmsMessageReceiver.ThreadContextLocal context = new XaTransactedJmsMessageReceiver.ThreadContextLocal();
JmsThreadContext ctx = context.getContext();
message = ctx.consumer.receive(timeout);
logger.debug("Message : " + message); // showing as a null here
Update :
When I open message on MQ Explorer, Message data bytes shows like below
00000 3C 00 3F 00 78 00 6D 00--6C 00 20 00 76 00 65 00 |����������������|
00010 72 00 73 00 69 00 6F 00--6E 00 3D 00 22 00 31 00 |����������������|
Messages are reading if message like this, then is it reading
00000 3C 3F 78 6D 6C 20 76 65--72 73 69 6F 6E 3D 22 31 |<?xml version="1|
00010 2E 30 22 20 65 6E 63 6F--64 69 6E 67 3D 22 75 74 |.0" encoding="ut|
This looks like MQ related issue, Can some help how the message formatting get changed?
Added Image:
I am a Windows user and my application accepts zip files.
I realized that when the user compresses files with the built in zip compressor in Mac OSX, it results in an extra folder titled "__MACOSX" created in the extracted zip.
I need to handle this folder(__MACOSX) in my application. I just want to know what is in the hidden __MACOSX directory. Is it empty or does it contain some files? And if it contains files, then how many files does it contain? If there are files, is the number of files fixed? What kind of files are there (empty/non-empty etc.)? Need full info.
It's simple to check in Mac OS but I don't have a Mac so I can't figure out what is there in this folder. I searched but couldn't find the answer.
I zipped a folder "Pack" containing the following:
Readme.txt
Image.jpg
Script.sh
Sound.m4a
What's inside "__MACOSX":
Pack (folder)
Pack/.Readme.txt (file)
Pack/.Image.jpg (file)
Pack/.Script.sh (file)
So it seems "__MACOSX" contains a replication of the folder structure being zipped, with hidden files starting with a dot, instead of the real files. However, not all files are there, so it might be difficult to predict how many files (in my test, the real file Sound.m4a don't have a .Sound.m4a equivalent.)
Those "hidden" files are not empty, they are binary files holding metadata.
Why don't you just ignore this "__MACOSX" folder, and delete it, instead of processing it?
Just to add to #Yoric's answer, the hidden files are written in the AppleDouble format, which is originally used to store additional metadata on Unix systems.
Example (GraCoL 2013 ICC):
unzip -d swop SWOP2013_and_GRACoL2013_ICC_Profiles.zip
file swop/__MACOSX/._GRACoL2013_CRPC6.icc
# swop/__MACOSX/._GRACoL2013_CRPC6.icc: AppleDouble encoded Macintosh file
Even without knowing the file format (documented in RFC 1740), you can more or less figure out what is saying. In our case:
$ hexdump -C swop/__MACOSX/._GRACoL2013_CRPC6.icc
00000000 00 05 16 07 00 02 00 00 4d 61 63 20 4f 53 20 58 |........Mac OS X|
00000010 20 20 20 20 20 20 20 20 00 02 00 00 00 09 00 00 | ........|
00000020 00 32 00 00 00 c2 00 00 00 02 00 00 00 f4 00 00 |.2..............|
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000050 00 00 00 00 41 54 54 52 00 00 00 00 00 00 00 f4 |....ATTR........|
00000060 00 00 00 98 00 00 00 5c 00 00 00 00 00 00 00 00 |.......\........|
00000070 00 00 00 00 00 00 00 01 00 00 00 98 00 00 00 5c |...............\|
00000080 00 00 15 63 6f 6d 2e 61 70 70 6c 65 2e 71 75 61 |...com.apple.qua|
00000090 72 61 6e 74 69 6e 65 00 71 2f 30 30 30 31 3b 35 |rantine.q/0001;5|
000000a0 32 33 39 64 33 32 37 3b 47 6f 6f 67 6c 65 5c 78 |239d327;Google\x|
000000b0 32 30 43 68 72 6f 6d 65 2e 61 70 70 3b 31 33 41 |20Chrome.app;13A|
000000c0 39 30 46 46 32 2d 34 43 45 41 2d 34 35 37 33 2d |90FF2-4CEA-4573-|
000000d0 38 37 45 32 2d 32 35 33 41 37 30 35 38 30 34 39 |87E2-253A7058049|
000000e0 44 7c 63 6f 6d 2e 67 6f 6f 67 6c 65 2e 43 68 72 |D|com.google.Chr|
000000f0 6f 6d 65 00 |ome.|
000000f4
We are looking at an instance of the com.apple.quarantine extended attribute, which in this case says the file was downloaded with Chrome. Well, at least it was the case for whoever packed this zip file.
I am attempting to remove quarantine from file permissions on osx; a similar starting point as this question.
However, when attempting the solution that worked there, I get:
$ xattr -d com.apple.metadata:kMDItemWhereFroms myFile.pem
xattr: [Errno 13] Permission denied: 'myFile.pem'
When I follow this up by applying sudo, I get:
$ sudo xattr -d com.apple.metadata myFile.pem
xattr: [Errno 2] No such file or directory: 'myFile.pem'
Anyone know what's going on here? My file clearly exists and I am unsure of what to try next.
Edit
Attempting to delete explicitly specified attribute still gives:
sudo xattr -d com.apple.metadata:kMDItemWhereFroms myFile.pem
xattr: [Errno 2] No such file or directory: 'myFile.pem'
In your second example you don't actually have a metadata attribute, i.e.
sudo xattr -d com.apple.metadata:kMDItemWhereFroms myFile.pem
Try listing the extended attributes first before removing them, which you can do via:
xattr -l myFile.pem
Then you should know which ones to delete/change in order to get the file unquarantined.
For example, if I do this on a .dmg file I see:
[~/Downloads]:;xattr -l yahoomessenger_3.0.2build235554.dmg
com.apple.diskimages.fsck:
00000000 6A EE 43 E7 65 3C 02 09 FC CC FD EE C5 BA 6F 0C |j.C.e<........o.|
00000010 F4 8B 4F 34 |..O4|
00000014
com.apple.diskimages.recentcksum: i:9292877 on 0B331FCB-4DC8-39B4-A12B-3A71BF73FD03 # 1341527466 - CRC32:$BA65FD8D
com.apple.metadata:kMDItemDownloadedDate:
00000000 62 70 6C 69 73 74 30 30 A1 01 33 41 B8 ED A2 37 |bplist00..3A...7|
00000010 EA AD 71 08 0A 00 00 00 00 00 00 01 01 00 00 00 |..q.............|
00000020 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 |................|
00000030 00 00 00 00 13 |.....|
00000035
com.apple.metadata:kMDItemWhereFroms:
00000000 62 70 6C 69 73 74 30 30 A1 01 5F 10 42 68 74 74 |bplist00.._.Bhtt|
00000010 70 3A 2F 2F 78 70 2E 79 69 6D 67 2E 63 6F 6D 2F |p://xp.yimg.com/|
00000020 67 6A 2F 6D 73 67 72 2F 6D 61 63 2F 79 61 68 6F |gj/msgr/mac/yaho|
00000030 6F 6D 65 73 73 65 6E 67 65 72 5F 33 2E 30 2E 32 |omessenger_3.0.2|
00000040 62 75 69 6C 64 32 33 35 35 35 34 2E 64 6D 67 08 |build235554.dmg.|
00000050 0A 00 00 00 00 00 00 01 01 00 00 00 00 00 00 00 |................|
00000060 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000070 4F |O|
00000071
com.apple.quarantine: 0002;533d6ab7;Safari;79D54B00-8D85-4DF7-BB49-AA9C69B9B6E9
So I see com.apple.quarantine and com.apple.metadata:kMDItemWhereFroms. I think just removing the quarantine attribute should do it.
And I see the edits you did, try simply doing xattr -d com.apple.quarantine /explicit/path/to/myFile.pem (i.e. pass along the exact filepath to the tool). No need to do sudo here.
I have a script that collects a bunch of file system object information (hashes, dates, etc) and stores it in a MySQL database (one row per object).
The script is running in Bash in Mac OS X 10.10.4 (MBP).
I would like to store the HFS+ Extended Attributes in the database as well. xattr gives output as shown below, I would like to dump the hex and formatting text leaving just the attribute name and the ASCII value. This means not just dumping the line numbers, hex, and | formatting characters but also concatenate the value onto one line per attribute name with the attribute name prepended.
Note that each object (file/folder) may have multiple attributes and the attribute names are not defined.
Take this input:
$xattr -l wordpress-3.9.6.zip
com.apple.metadata:kMDItemWhereFroms:
00000000 62 70 6C 69 73 74 30 30 A2 01 02 5F 10 29 68 74 |bplist00..._.)ht|
00000010 74 70 73 3A 2F 2F 77 6F 72 64 70 72 65 73 73 2E |tps://wordpress.|
00000020 6F 72 67 2F 77 6F 72 64 70 72 65 73 73 2D 33 2E |org/wordpress-3.|
00000030 39 2E 36 2E 7A 69 70 5F 10 2F 68 74 74 70 73 3A |9.6.zip_./https:|
00000040 2F 2F 77 6F 72 64 70 72 65 73 73 2E 6F 72 67 2F |//wordpress.org/|
00000050 64 6F 77 6E 6C 6F 61 64 2F 72 65 6C 65 61 73 65 |download/release|
00000060 2D 61 72 63 68 69 76 65 2F 08 0B 37 00 00 00 00 |-archive/..7....|
00000070 00 00 01 01 00 00 00 00 00 00 00 03 00 00 00 00 |................|
00000080 00 00 00 00 00 00 00 00 00 00 00 69 |...........i|
0000008c
com.apple.quarantine: 0001;55701556;Google Chrome.app;8AD80928-CB48-48EA-8A1B-EC4B0BE656A9
And make it look like this:
com.apple.metadata:kMDItemWhereFroms: bplist00..._.)https://wordpress.org/wordpress-3.9.6.zip_./https://wordpress.org/download/release-archive/..7...............................i
com.apple.quarantine: 0001;55701556;Google Chrome.app;8AD80928-CB48-48EA-8A1B-EC4B0BE656A9
Thanks for any help
MC
xattr is not very customizable; it's meant more for human browsing than scripted use. You're better off using another language. Here's an example in Python:
import xattr
x = xattr.xattr('wordpress-3.9.6.zip')
for name, value in x:
print name, repr(x[name])
You may want to drop the call to repr (or use a different wrapper around x[name]), depending on the desired output.
Note that you almost certainly do not want the . from the ASCII output of the xattr program, since they represent any non-printable ASCII character.