Removing null bytes inline from file on OS X - macos

I have a set of files which have null bytes roughly at every second byte slot. Further, they have the byte sequence FF FE at the beginning, which needs to be removed. I was given the files (they are actually router configuration files, so plain text), and I have no idea how those bytes got in there. However, the file looks like this, which I found for example here or here, :
FF FE 65 00 6E 00 61 00 62 6C 00 6C 65 00 0D 00
0A 00 63 00 6F 00 6E 00 66 00 69 00 37 00 75 00
72 00 65 00 20 00 74 00 65 00 72 00 6D 00 69 00
Think you can imagine how it goes on. I tried various things to remove the null bytes and the first 2 bytes as well:
sed -i.bak 's/\x00//g' R2.txt does nothing.
LC_ALL=C sed -i.bak 's/\x00//g' R2.txt does nothing.
LC_ALL=C tr < R2.txt -d '\000' > R2-test.txt works, but is not in line.
LC_ALL=C sed -i.bak $'s/\x00//g' R2.txt complains about sed: 1: "s/": unterminated substitute pattern
So... my question is, how to remove the null bytes the first two bytes (FF FE) from this file, inline, on OSX?
Thanks.

you said " null bytes roughly at every second byte slot". This is a clear indication of the file being encoded with "charset=utf-16le" and you will need to convert it to utf-8 , ascii ..etc. try something like the below:
iconv -f "current-encoding" -t "desired-encoding" infile.txt > outfile.txt

Related

What's inside the __MACOSX hidden folder in zip files created in Mac OS

I am a Windows user and my application accepts zip files.
I realized that when the user compresses files with the built in zip compressor in Mac OSX, it results in an extra folder titled "__MACOSX" created in the extracted zip.
I need to handle this folder(__MACOSX) in my application. I just want to know what is in the hidden __MACOSX directory. Is it empty or does it contain some files? And if it contains files, then how many files does it contain? If there are files, is the number of files fixed? What kind of files are there (empty/non-empty etc.)? Need full info.
It's simple to check in Mac OS but I don't have a Mac so I can't figure out what is there in this folder. I searched but couldn't find the answer.
I zipped a folder "Pack" containing the following:
Readme.txt
Image.jpg
Script.sh
Sound.m4a
What's inside "__MACOSX":
Pack (folder)
Pack/.Readme.txt (file)
Pack/.Image.jpg (file)
Pack/.Script.sh (file)
So it seems "__MACOSX" contains a replication of the folder structure being zipped, with hidden files starting with a dot, instead of the real files. However, not all files are there, so it might be difficult to predict how many files (in my test, the real file Sound.m4a don't have a .Sound.m4a equivalent.)
Those "hidden" files are not empty, they are binary files holding metadata.
Why don't you just ignore this "__MACOSX" folder, and delete it, instead of processing it?
Just to add to #Yoric's answer, the hidden files are written in the AppleDouble format, which is originally used to store additional metadata on Unix systems.
Example (GraCoL 2013 ICC):
unzip -d swop SWOP2013_and_GRACoL2013_ICC_Profiles.zip
file swop/__MACOSX/._GRACoL2013_CRPC6.icc
# swop/__MACOSX/._GRACoL2013_CRPC6.icc: AppleDouble encoded Macintosh file
Even without knowing the file format (documented in RFC 1740), you can more or less figure out what is saying. In our case:
$ hexdump -C swop/__MACOSX/._GRACoL2013_CRPC6.icc
00000000 00 05 16 07 00 02 00 00 4d 61 63 20 4f 53 20 58 |........Mac OS X|
00000010 20 20 20 20 20 20 20 20 00 02 00 00 00 09 00 00 | ........|
00000020 00 32 00 00 00 c2 00 00 00 02 00 00 00 f4 00 00 |.2..............|
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000050 00 00 00 00 41 54 54 52 00 00 00 00 00 00 00 f4 |....ATTR........|
00000060 00 00 00 98 00 00 00 5c 00 00 00 00 00 00 00 00 |.......\........|
00000070 00 00 00 00 00 00 00 01 00 00 00 98 00 00 00 5c |...............\|
00000080 00 00 15 63 6f 6d 2e 61 70 70 6c 65 2e 71 75 61 |...com.apple.qua|
00000090 72 61 6e 74 69 6e 65 00 71 2f 30 30 30 31 3b 35 |rantine.q/0001;5|
000000a0 32 33 39 64 33 32 37 3b 47 6f 6f 67 6c 65 5c 78 |239d327;Google\x|
000000b0 32 30 43 68 72 6f 6d 65 2e 61 70 70 3b 31 33 41 |20Chrome.app;13A|
000000c0 39 30 46 46 32 2d 34 43 45 41 2d 34 35 37 33 2d |90FF2-4CEA-4573-|
000000d0 38 37 45 32 2d 32 35 33 41 37 30 35 38 30 34 39 |87E2-253A7058049|
000000e0 44 7c 63 6f 6d 2e 67 6f 6f 67 6c 65 2e 43 68 72 |D|com.google.Chr|
000000f0 6f 6d 65 00 |ome.|
000000f4
We are looking at an instance of the com.apple.quarantine extended attribute, which in this case says the file was downloaded with Chrome. Well, at least it was the case for whoever packed this zip file.

Conflicting answers when I try to find out endian-ness of my Macbook Pro

I have a Macbook Pro and am getting contradictory answers when I try to determine its endian-ness.
Method 1
python -c "import sys;print sys.byteorder" tells me I am on a little endian system
Method 2
I have a text file. I used iconv to convert it into UTF16. Its supposed to detect the endianness of the computer and convert it into that format. So here I go:
iconv -f utf-8 -t utf-16 file.txt > utf16.txt
file utf16.txt
utf16.txt: Big-endian UTF-16 Unicode English text
vi utf16.txt works and hexdump -C utf16.txt shows:
00000000 fe ff 00 33 00 39 00 38 00 31 00 36 00 30 00 38 |...3.9.8.1.6.0.8|
00000010 00 09 00 54 00 69 00 61 00 20 00 4a 00 75 00 61 |...T.i.a. .J.u.a|
00000020 00 6e 00 61 00 20 00 52 00 69 00 76 00 65 00 72 |.n.a. .R.i.v.e.r|
00000030 00 09 00 54 00 69 00 61 00 20 00 4a 00 75 00 61 |...T.i.a. .J.u.a|
00000040 00 6e 00 61 00 20 00 52 00 69 00 76 00 65 00 72 |.n.a. .R.i.v.e.r|
00000050 00 09 00 52 00 69 00 6f 00 20 00 54 00 69 00 61 |...R.i.o. .T.i.a|
00000060 00 6a 00 75 00 61 00 6e 00 61 00 2c 00 52 00 69 |.j.u.a.n.a.,.R.i|
00000070 00 6f 00 20 00 54 00 69 00 6a 00 75 00 61 00 6e |.o. .T.i.j.u.a.n|
00000080 00 61 00 2c 00 52 00 ed 00 6f 00 20 00 54 00 69 |.a.,.R...o. .T.i|
00000090 00 6a 00 75 00 61 00 6e 00 61 00 2c 00 54 00 69 |.j.u.a.n.a.,.T.i|
if I convert it to little-endian and manually insert a BOM like this:
( printf "\xff\xfe" ; iconv -f utf-8 -t utf-16le file.txt ) > UTF16LEBOM.txt
file UTF16LEBOM.txt
UTF16LEBOM.txt: Little-endian UTF-16 Unicode English text
vi UTF16LEBOM.txt works
and hexdump -C UTF16LEBOM.txt shows
00000000 ff fe 33 00 39 00 38 00 31 00 36 00 30 00 38 00 |..3.9.8.1.6.0.8.|
00000010 09 00 54 00 69 00 61 00 20 00 4a 00 75 00 61 00 |..T.i.a. .J.u.a.|
00000020 6e 00 61 00 20 00 52 00 69 00 76 00 65 00 72 00 |n.a. .R.i.v.e.r.|
00000030 09 00 54 00 69 00 61 00 20 00 4a 00 75 00 61 00 |..T.i.a. .J.u.a.|
00000040 6e 00 61 00 20 00 52 00 69 00 76 00 65 00 72 00 |n.a. .R.i.v.e.r.|
00000050 09 00 52 00 69 00 6f 00 20 00 54 00 69 00 61 00 |..R.i.o. .T.i.a.|
00000060 6a 00 75 00 61 00 6e 00 61 00 2c 00 52 00 69 00 |j.u.a.n.a.,.R.i.|
00000070 6f 00 20 00 54 00 69 00 6a 00 75 00 61 00 6e 00 |o. .T.i.j.u.a.n.|
00000080 61 00 2c 00 52 00 ed 00 6f 00 20 00 54 00 69 00 |a.,.R...o. .T.i.|
00000090 6a 00 75 00 61 00 6e 00 61 00 2c 00 54 00 69 00 |j.u.a.n.a.,.T.i.|
From this link:
The other approach is to include a magic number, such as 0xFEFF,
before every piece of data. If you read the magic number and it is
0xFEFF, it means the data is in the same format as your machine, and
all is well.
If you read the magic number and it is 0xFFFE (it is backwards), it
means the data was written in a format different from your own. You'll
have to translate it.
Who is right and why am I getting contradictory answers?
"Endian-ness of my Macbook Pro" means nothing. You need to be more specific; different applications will have different impressions. As you've just seen, you can arbitrarily encode bytes in a file. In the end, a series of bytes is just that, and files are ultimately just a series of bytes that can be read in either fashion. In the context of programming (Stack Overflow) what's important is knowing a) Whether the input you are getting is in Big Endian or Little Endian, and b) Whether the output you send should be in Big Endian or Little Endian.
If your question is the conventional reading of files, the answer is usually Little Endian. But, for example, network data tends to be Big Endian.

Remove 0 character

I have a text file that contains junk characters after some lines. As far as I know this is character 0 (not '0') although I am not sure. When I open the file in vi one such line looks like this:
ESH6^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#
I am only interested in the "ESH6" part of that line, I do not want the rest. If I do a hexdump -C that line looks like this (hexdump -C foo.txt | head -n 2, the "49705" is part of another line):
00000000 45 53 48 36 00 00 00 00 00 00 00 00 00 00 00 00 |ESH6............|
00000010 00 00 00 00 0a 0a 34 39 37 30 35 0a 0a 0a 0a 45 |......49705....E|
How can I remove all instances of this character from the file?
Use the tr command:
tr -d '\0' < foo.txt > output
This deletes (-d) all null bytes ('\0') from the standard input and writes everything else to standard output. The tr command is a pure filter; it only reads standard input and only writes to standard output.

How can I add OS X "tags" to files programmatically?

Since Mavericks, OS X has had the ability to tag & colour files in Finder.
Is there any way to add tags to files through Cocoa APIs or via a shell command?
Check out tag, "a command line tool to manipulate tags on Mac OS X 10.9 Mavericks files, and to query for files with those tags". The GitHub repository has installation instructions (there are Homebrew and MacPorts packages).
Sorry for adding another answer, but the one related to setting Label colors was pretty long already. Here is an excerpt from a python script that I use to set the User Tags. It seems to work to make things searchable, but not sure if the tags will show up correctly. Usage is basically:
tagfile.py "Tag Name" FileOrFolderName
Code below.
#! /usr/bin/env python
# -*- coding: utf-8 -*-
""" Write tags to file
Usage:
tagfile.py "TagName" FileName1 FileName2
You can use wildcards for the file name. Use quotes if spaces in tags.
To check if it worked, use xattr -l FileName
"""
import sys
import subprocess
def writexattrs(F,TagList):
""" writexattrs(F,TagList):
writes the list of tags to three xattr fields on a file-by file basis:
"kMDItemFinderComment","_kMDItemUserTags","kMDItemOMUserTags
Uses subprocess instead of xattr module. Slower but no dependencies"""
Result = ""
plistFront = '<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><array>'
plistEnd = '</array></plist>'
plistTagString = ''
for Tag in TagList:
plistTagString = plistTagString + '<string>{}</string>'.format(Tag.replace("'","-"))
TagText = plistFront + plistTagString + plistEnd
OptionalTag = "com.apple.metadata:"
XattrList = ["kMDItemFinderComment","_kMDItemUserTags","kMDItemOMUserTags"]
for Field in XattrList:
XattrCommand = 'xattr -w {0} \'{1}\' "{2}"'.format(OptionalTag + Field,TagText.encode("utf8"),F)
if DEBUG:
sys.stderr.write("XATTR: {}\n".format(XattrCommand))
ProcString = subprocess.check_output(XattrCommand, stderr=subprocess.STDOUT,shell=True)
Result += ProcString
return Result
DEBUG = False
if __name__ == "__main__":
if len(sys.argv) < 3:
print __doc__
else:
TagList = [ sys.argv[1] ]
# print TagList
# Or you can hardwire your tags here
# TagList = ['Orange','Green']
FileList = sys.argv[2:]
for FileName in FileList:
writexattrs(FileName, TagList)
I add this answer, because OP asked for a shell script and tagged it bash. I wrote this Automator service, which tags the selected file with the tags of another file. I have added comments to outline the use of bash's interaction with the tags and colours using bash script.
 
Basics
In scripts both OpenMeta and Mavericks tags can be accessed with the command xattr. Using it without modifiers, $ xattr [file], gives a list of set attributes. $ xattr -h gives a nice guide to usage.
Mavericks' tags are in com.apple.metadata:_kMDItemUserTags, while OpenMeta tags can be in a variety of attributes. Amongst others com.apple.metadata:kOMUserTags, org.openmetainfo:kMDItemOMUserTags and org.openmetainfo:kOMUserTags.
Mavericks handles colours and tags in different attributes, by placing tags in _kMDItemUserTags and colours in FinderInfo for every file. This is a bizarre choice, and it is one of the reasons Finder struggles under the pressure of tagging. If you have 800 files tagged kapow, each in a different folder, and you subsequently choose the colour blue for kapow, Finder has to find and alter attributes for every single file.
You can play around with the oddity by removing the com.apple.FinderInfo attribute from a tagged and coloured file: $ xattr -d com.apple.FinderInfo [file]. The colour will disappear in Finder listings, but the tag (and its colour) remains associated with the file.
 
Bash script to import tags from another file
In the script, the selected file(s) in Finder is/are saved to the variable $tagless, and the chosen supplier of tags is $tagfull.
TAGFULID=${##}
TAGFUL=${!TAGFULID}
## Use xattr to read all existing tags:
ATTRS=$(xattr "$TAGFUL")
for f in "$#" ## For every selected file in Finder, do:
do
if("$TAGFUL"="$f") ## Is the supplier of tags is amongst the selected files?
then
break
fi
if [[ "$ATTRS" == *kMDItemUserTags* ]] ## Are there tags?
then
## Load tags:
TAGS=$(xattr -px com.apple.metadata:_kMDItemUserTags "$TAGFUL")
## Write tags:
xattr -wx com.apple.metadata:_kMDItemUserTags "$TAGS" "$f"
fi
if [[ "$ATTRS" == *FinderInfo* ]] ## Are there colours?
then
## Load colour:
FINDERINFO=$(xattr -px com.apple.FinderInfo "$TAGFUL")
## Write colour:
xattr -wx com.apple.FinderInfo "$FINDERINFO" "$f"
fi
done
You could give this a shot:
xattr -w com.apple.metadata:_kMDItemUserTags '<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><array><string>Orange</string><string>Red</string></array></plist>' $currentFile
You'll want to replace $currentFile with the file you'd like to add tags to, and change
<string>Orange</string><string>Red</string>
to a list of whatever tags you want to add.
In Apple's What's New in OS X it states that NSURL handles tags, and the Common File System Resource_Keys gives the required key as NSURLTagNamesKey and states its value is just an array of strings.
This does not cover tags, but for changing label colors, one way to do it is through a command like this:
xattr -wx com.apple.FinderInfo \
0000000000000000000400000000000000000000000000000000000000000000 myfile.txt
The 04 buried in the middle is setting the file color.
Here is a python script which wraps that command lets you set the tag color on a file or series of files:
import sys
import subprocess
def colorizeFile(ColorName,FileName):
ReverseTable = {
"clear" : "01",
"gray" : "03",
"green" : "04",
"purple" : "06",
"blue" : "09",
"yellow" : "0A",
"red" : "0C",
"orange" : "0E",
"c" : "01",
"a" : "03",
"g" : "04",
"p" : "06",
"b" : "09",
"y" : "0A",
"r" : "0C",
"o" : "0E",
}
HexString = 18*"0" + ReverseTable.get(ColorName) + 44*"0"
Xcommand = 'xattr -wx com.apple.FinderInfo {0} {1}'.format(HexString,FileName)
ProcString = subprocess.check_call(Xcommand, stderr=subprocess.STDOUT,shell=True)
if __name__ == "__main__":
if len(sys.argv)<3:
sys.stderr.write(__doc__.format(sys.argv[0]))
else:
Cname = sys.argv[1]
Flist = sys.argv[2:]
for File in Flist:
colorizeFile(Cname.lower(),File)
sys.stderr.write("## Colorized {0} file(s) as {1}\n".format(len(Flist),Cname))
Usage is:
labelcolor.py [color] *.jpg
where [color] is a name or abbreviation as defined below:
clear (c), grAy (a), green (g), purple (p),
blue (b), yellow (y), red (r), orange (o)
The OpenMeta framework is a third-party standard for adding metadata to OS X files using extended attributes. It is used by a number of third-party applications.
Or you can use the XATTR command to manipulate the extended attributes via command line.
Starting with Mavericks, it is possible to get and set color tags in Cocoa, using NSURL.
NSURL has a slew of properties that can be set or read, through the respective setResourceValue:forKey:error: and getResourceValue:forKey:error: methods.
Using the NSURLLabelNumberKey key, you can set the color tags, as follows:
NSURL *fileURL = [NSURL fileURLWithPath:#"/Users/[username]/Documents/[some_file]"];
NSError *resourceError;
if (![fileURL setResourceValue:#(2) forKey:NSURLLabelNumberKey error:&resourceError]) {
NSLog(#"Error while setting file resource: %#", [resourceError localizedDescription]);
}
If this is executed on a file that only has one color, then it clears the current color, and sets the specified color. However, if multiple colors are already set on the file, then it does not clear the existing colors before setting the specified color.
Here is the value-color mapping (on El Capitan):
#(0): None
#(1): Grey
#(2): Green
#(3): Purple
#(4): Blue
#(5): Yellow
#(6): Red
#(7): Orange
I have not been able to set a tag using NSURLLabelColorKey. Here is my experience on El Capitan, with the keys related to 'tags' (Colors):
NSURLLabelNumberKey: can be read/set successfully, with numbers 0-7. Any other number will return an error. If there are multiple tags set, then this will return the index of the first color that is set, as it searches numerically through the indexes 1 through 7. Although you can clear a color in Finder by clicking on the color, programmatically setting a color that is already set does not clear that color.
NSURLLabelColorKey: returns nil, even when a color tag is set for a file. Setting a value with this key has no effect.
NSURLTagNamesKey: returns an array of the color names for the tags that are set.
In Ask Different
With multiple answers, one of which is accepted:
Possible to tag a folder via terminal? (2013-11-15)
Here in Stack Overflow the question arose slightly earlier (2013-11-01) so I'll add my answer here.
openmeta
Open source at https://code.google.com/p/openmeta/source/browse/trunk/trunk/openmeta
The openmeta command appears to take a dual attribute approach, working with both:
com.apple.metadata:kMDItemOMUserTags
com.apple.metadata:_kMDItemUserTags
Example usage
sh-3.2$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.9.5
BuildVersion: 13F1096
sh-3.2$ uname -a
Darwin gpes3e-gjp4.local 13.4.0 Darwin Kernel Version 13.4.0: Wed Mar 18 16:20:14 PDT 2015; root:xnu-2422.115.14~1/RELEASE_X86_64 x86_64
sh-3.2$ date
Sun 26 Jul 2015 08:00:23 BST
sh-3.2$ rm ~/Desktop/test.txt
sh-3.2$ touch ~/Desktop/test.txt
sh-3.2$ xattr -l ~/Desktop/test.txt
sh-3.2$ ./openmeta
openmeta version 0.1 by Tom Andersen code.google.com/p/openmeta/
Usage: openmeta [options] -p PATH[s]
Note that commas are to be used nowhere - tag lists use quotes for two word tags in output
example (list tags and ratings): openmeta -p PATH
example (list tags and ratings multiple): openmeta -p PATH PATH
example (list tags): openmeta -t -p PATH[s]
example (add tags): openmeta -a foo bar -p PATH[s]
example (add tags with spaces): openmeta -a "three word tag" "foo bar" -p PATH[s]
example (set tags): openmeta -s foo bar -p PATH[s]
example (clear all tags): openmeta -s -p PATH[s]
example (set managed): openmeta -m Y -p PATH[s]
example (set rating 0 - 5 stars): openmeta -r 3.5 -p PATH[s]
example (print rating): openmeta -r -p PATH[s]
example (clear rating): openmeta -r 0.0 -p PATH[s]
example (lousy rating): openmeta -r 0.1 -p PATH[s]
sh-3.2$ ./openmeta -a kerfuffle -p ~/Desktop/test.txt
kerfuffle /Users/gjp22/Desktop/test.txt
sh-3.2$ ./openmeta -p ~/Desktop/test.txt
/Users/gjp22/Desktop/test.txt
tags: kerfuffle
rating: none found
sh-3.2$ xattr -l ~/Desktop/test.txt
com.apple.metadata:kMDItemOMUserTagTime:
00000000 62 70 6C 69 73 74 30 30 33 41 BB 64 BD 3C D4 95 |bplist003A.d.<..|
00000010 F2 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00 |................|
00000020 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000030 00 11 |..|
00000032
com.apple.metadata:kMDItemOMUserTags:
00000000 62 70 6C 69 73 74 30 30 A1 01 59 6B 65 72 66 75 |bplist00..Ykerfu|
00000010 66 66 6C 65 08 0A 00 00 00 00 00 00 01 01 00 00 |ffle............|
00000020 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 |................|
00000030 00 00 00 00 00 14 |......|
00000036
com.apple.metadata:_kMDItemUserTags:
00000000 62 70 6C 69 73 74 30 30 A1 01 5B 6B 65 72 66 75 |bplist00..[kerfu|
00000010 66 66 6C 65 0A 30 08 0A 00 00 00 00 00 00 01 01 |ffle.0..........|
00000020 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 |................|
00000030 00 00 00 00 00 00 00 16 |........|
00000038
kOM109SyncDone:
00000000 62 70 6C 69 73 74 30 30 09 08 00 00 00 00 00 00 |bplist00........|
00000010 01 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 |................|
00000020 00 00 00 00 00 00 00 00 00 09 |..........|
0000002a
sh-3.2$
Limitations of other utilities
Apple Finder, for example.
After using Finder to remove the kerfuffle tag, kerfuffle remains as an OpenMeta tag:
sh-3.2$ date ; xattr -l ~/Desktop/test.txt
Sun 26 Jul 2015 08:02:13 BST
com.apple.metadata:kMDItemOMUserTagTime:
00000000 62 70 6C 69 73 74 30 30 33 41 BB 64 BD 3C D4 95 |bplist003A.d.<..|
00000010 F2 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00 |................|
00000020 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000030 00 11 |..|
00000032
com.apple.metadata:kMDItemOMUserTags:
00000000 62 70 6C 69 73 74 30 30 A1 01 59 6B 65 72 66 75 |bplist00..Ykerfu|
00000010 66 66 6C 65 08 0A 00 00 00 00 00 00 01 01 00 00 |ffle............|
00000020 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 |................|
00000030 00 00 00 00 00 14 |......|
00000036
com.apple.metadata:_kMDItemUserTags:
00000000 62 70 6C 69 73 74 30 30 A0 08 00 00 00 00 00 00 |bplist00........|
00000010 01 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 |................|
00000020 00 00 00 00 00 00 00 00 00 09 |..........|
0000002a
kOM109SyncDone:
00000000 62 70 6C 69 73 74 30 30 09 08 00 00 00 00 00 00 |bplist00........|
00000010 01 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 |................|
00000020 00 00 00 00 00 00 00 00 00 09 |..........|
0000002a
sh-3.2$
Understanding those limitations
With attention to domains and naming conventions: Developer thoughts on adopting OpenMeta – Ironic Software (2009-03, and now in the Internet Archive Wayback Machine) reminds us that com.apple.metadata was for use by Apple when OpenMeta (a project not in the apple.com domain) began the Apple-oriented com.apple.metadata:kMDItemOMUserTags approach.
So I should not expect Apple software to gain or maintain compatibility with both approaches to tagging.
Edge cases
In some cases it may be desirable to remove Apple-oriented com.apple.metadata:_kMDItemUserTags tags without removing OpenMeta-oriented com.apple.metadata:kMDItemOMUserTags tags.
However, doing so – programmatically – is probably beyond the scope of the question asked by #nacross.
I just implemented OSXMETADATA in Python to copy OSX finder tags
https://pypi.org/project/osxmetadata/

Printing string representations of xattr hex output

I'm trying to write a script to extract the original download URL from disk images downloaded with Safari on OS X using xattr, so that I can rename them but still easily obtain their original names for reference.
This command prints the hex representation of the URL that the given file was downloaded from, as an example:
xattr -p com.apple.metadata:kMDItemWhereFroms *.dmg
gives
62 70 6C 69 73 74 30 30 A1 01 5F 10 4F 68 74 74
70 3A 2F 2F 61 64 63 64 6F 77 6E 6C 6F 61 64 2E
61 70 70 6C 65 2E 63 6F 6D 2F 4D 61 63 5F 4F 53
5F 58 2F 6D 61 63 5F 6F 73 5F 78 5F 31 30 2E 36
2E 31 5F 62 75 69 6C 64 5F 31 30 62 35 30 34 2F
30 34 31 35 30 37 33 61 2E 64 6D 67 08 0A 00 00
00 00 00 00 01 01 00 00 00 00 00 00 00 02 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 5C
The URL starts at the 14th byte (if I counted correctly) and is NULL terminated. How can I format this string so that I get a string output as follows:
http://adcdownload.apple.com/Mac_OS_X/mac_os_x_10.6.1_build_10b504/0415073a.dmg
(don't worry, this link doesn't work unless you're logged in to ADC)
...essentially, the same thing Finder will display in Get Info. I tried piping xattr's output to xxd but I'm not sure how to specify the offset so the string starts at the right place.
So, after looking at the binary data returned by xattr -p, I realized that it was actually a binary plist... hence "bplist" at the front of the data. For some reason I didn't notice this before, but in light of this, here's a proper solution that should work on every OS X from 10.5 to 10.8.
To avoid duplication, I'll link to the source instead of pasting it: https://github.com/jakepetroules/wherefrom

Resources