I have url links to image files I want to retrieve from the internet.
I can download the files using curl without issue using:
curl "https://...web address..." > myfileName;
The image files are of various types, some .bmp some .jpg etc. I have been using sip in Terminal on Mac osx to convert each to .png files using:
sips -s format png downloadFileName --out newFileName.png
This works well on files I've saved as downloadedFileName regardless of the starting file type.
As I have many files to process I wanted to pipe the output of the curl download directly into sips, without saving an intermediate file.
I tried the following (which combines my two working steps without the intermediate file name):
curl "https://...web address..." | sips -s format png --out fileName.png
And get a no file error: Error 4: no file was specified.
I've searched the sip man pages but cannot find a reference for piped input and have been unable to find a useful answer searching SO or google.
Is there a way to process an image downloaded using curl directly in sips without first saving the file?
I do not necessarily need the solution to use a pipe, or even be on one line. I have a script that will cycle through a few thousand urls and simply want to avoid saving lots of files that will be deleted a line later.
I should add, I do not necessarily need to use sips either. However, any solution must be able to handle image files of unknown type (which sips does admirably) as no file extension is present on the files.
Thanks
I don't have sips installed but its
manpage indicates that it cannot read
from stdin. However, if you use Bash or ZSH (MacOS default now) you
can use process substitution, in this example I use convert which is
a part of ImageMagick and can convert different image types too:
$ convert <(curl -s https://i.kym-cdn.com/entries/icons/mobile/000/018/012/this_is_fine.jpg) this_is_fine.png
$ file this_is_fine.png
this_is_fine.png: PNG image data, 800 x 450, 8-bit/color RGB, non-interlaced
After doing that this_is_fine.png will be the only file in the
directory with no temporary files
Apparently sips only reads regular files which makes it impossible to use /dev/stdin or named pipes.
However, it is possible using the mature and feature-rich convert command:
$ curl -sL https://picsum.photos/200.jpg | convert - newFilename.png
$ file newFilename.png
newFilename.png: PNG image data, 200 x 200, 8-bit/color RGB, non-interlaced
(First install ImageMagick via brewinstall imagemagick or sudoportinstall ImageMagick.)
ImageMagick permits image data to be read and written from the standard streams STDIN (standard in) and STDOUT (standard out), respectively, using a pseudo-filename of -.
source, section STDIN, STDOUT, and file descriptors
Related
Can someone show me how to use the PostScript deletefile operator to delete the input file after GhostScript finishes converting the input file to a PDF file.
This appears to work for me, first creating the PDF file, then setting the permissions on the input file, and finally deleting the input file.
"C:/Program Files/gs/gs9.55.0/bin/gswin64c.exe" -q -sDEVICE#pdfwrite
-o "C:/Temp/Temp_0001.pdf"
-f "C:/Temp/Temp_0001.ps"
--permit-file-all=C:/Temp/Temp_0001.ps
-c (C:/Temp/Temp_0001.ps) deletefile
NOTE: Since I had to switch to Unix-style path separators (even though I am running this on Windows) for the permit-file-all and the deletefile, I decided to use the same convention for both the output and input files as well. Windows seems to be OK with that, and the convention was uniformly used for all paths/files.
I am creating a dmg file and getting following error while using the icns and png files..
Here is the command:
DeRez -only icns resources/test.icns > icns.rsrc
Error:
/Applications/Xcode.app/Contents/Developer/usr/bin/DeRez - SysError
-39 during open of resource file "resources/test.icns"
Mac OS version: macOS mojave 10.14.2
Please suggest.
Encountered this just now as well and noticed this question has gone unanswered, even though it's fairly simple.
Apparently error -39 means end of file, I'm assuming this means it wasn't able to find a valid portion of expected data. I'm guessing you (like myself) used sips -i someicon.icns to set the file's icon to itself? You should've gotten an error --addIcon is no longer supported which to me means that it didn't actually set the icon. So a subsequent DeRez call to get a .rsrc out of it will fail.
I didn't want to install any extra stuff but luckily I found another answer on here that seems to use only standard Xcode and/or macOS tools. In my case I was trying to set an AppleScript-exported .app icon, but the actual conversion part should be the same regardless.
So, all the steps to go from .png to an actual icon for an .app file:
cd /some/application/dir/some.app/Contents/Resources
cp /some/assets/dir/myicon.png ./
# Convert png to icns
# Keep in mind that sips can allegedly only handle 256x256, 512x512 and 1024x1024 PNGs for this
# Otherwise you need to do something like the following first:
# sips -z 256 256 myicon.png --out myicon_resized.png
# Then check if it came out good and run:
# mv -f myicon_resized.png myicon.png
# Now let's actually convert it and remove the png as it's no longer needed
sips -s format icns myicon.png --out myicon.icns
rm -fv myicon.png
# Apparently you can also specify commands of some sort in .rsrc files (this is the part I needed from the other SO answer)
# This is used in place of DeRez
echo "read 'icns' (-16455) \"myicon.icns\";" > myicon.rsrc
# Now just use Rez to append it to the special icon file in the root of the .app dir
Rez -append myicon.rsrc -o $'../../Icon\r'
# Write the metadata indicating we want to use the Icon\r file for the app's icon
# I could just use ../../ for the path but I like to be explicit
SetFile -a C ../../../some.app
# Also hide said file from Finder views
SetFile -a V $'../../Icon\r'
I haven't tried setting custom icons for a .dmg file, but I guess you already know how to "apply" the .rsrc to it anyways. =]
I can access the meta data property "owner url" thru Photoshop, but am hoping that there's a way to access it from the command line without having to open the file.
Does anyone know of a way to do this?
mdls doesn't list this particular metadata field.
There is no built-in command line tool to achieve this.
However, you can utilize exiftool, which is a platform-independent Perl library plus a command-line application for reading, writing and editing meta information in a wide variety of files.
Installation:
The guidelines for installing it on macOS can be found here. In summary:
Download the ExifTool OS X Package from the ExifTool home page.
(The file you download should be named ExifTool-11.17.dmg.)
Install as a normal OS X package.
(Open the disk image, double-click on the install package, and
follow the instructions.)
You can now run exiftool by typing exiftool in a Terminal window.
Processing a single file:
Reading the "owner url" via the command line:
Run the following command in a Terminal window:
$ exiftool -b −xmp:WebStatement ~/Desktop/path/to/image.psd
Note: the ~/Desktop/path/to/image.psd part in the command above should be replaced with a real image filepath.
This command will log the URL to the console only if the image metadata contains one. For instance:
https://www.example.com
Writing the "owner url" via the command line:
You can also write the "owner url" to a file by running the following command:
$ exiftool −xmp:WebStatement="https://www.foobar.com" ~/Desktop/path/to/image.psd
Note: As mentioned previously, the ~/Desktop/path/to/image.psd part in the command above should be replaced with a real image filepath, and the https://www.foobar.com part should be replaced with the actual URL you want to apply.
Processing multiple files:
Reading the "owner url" for multiple files via the command line:
If you wanted to read the "owner url" for all image files within a given folder, (including those in sub folders), and generate a JSON report you can run the following command:
$ exiftool -j -r −xmp:WebStatement ~/Desktop/path/to/folder/ -ext jpg -ext png -ext psd -ext tif > ~/Desktop/owner-urls.json
Breakdown of command (above):
-j - Use JSON formatting for output.
-r - Recursively process sub directories.
−xmp:WebStatement - Retrieve the WebStatement value, i.e. "owner url".
~/Desktop/path/to/folder/ - The path to the folder containing images (This should be replaced with a real path to a folder).
-ext jpg -ext png -ext psd -ext tif - The file extension(s) to process.
> ~/Desktop/owner-urls.json - Save the JSON output to file at the Desktop named owners-url.json.
I am using preview (that comes with OS X El Capitan) feature to convert a file form TIFF format into JPG for example. I expected the export process will include the original comments, but it doesn't happen (it applies also for the tag fields).
The generated JPG file has no comment
The compression and change image format work, but the META INFO such as comment or tags are not exported.
Any suggestion or workaround about how to include that information. I need to convert about 500 images so manually copy/paste doesn't work for me.
Updated Answer
In the light of your comments, I think the best way forward is to try and identify how/where the comments are stored for each platform (Windows vs macOS) and then to decide which method you want to use going forward.
macOS Finder/Spotlight comments will not be legible on Windows, so if you want Windows compatibility, you need to standardise on JPEG or EXIF comments.
I recommend using exiftool which you can install with homebrew, using:
brew install exiftool
Then I suggest you try extracting the comments from your files to see how/where they are stored:
exiftool -a image.jpg
will show you all tags in image.jpg. Your comments may be under:
comment - which is the JPEG comment, or
EXIF:UserComment - which is the EXIF comment
If you find your comments in the JPEG or the EXIF section, you can extract just the comments with:
exiftool -comment image.jpg # extract JPEG comment
exiftool -EXIF:UserComment image.jpg # extract EXIF UserComment
Add the option -s3 to suppress the field-names in the above to save having to parse them out.
Likewise, you can set the comments with:
exiftool -comment="FUNKY JPEG COMMENT" image.jpg # set JPEG comment
exiftool -EXIF:UserComment="FUNKY EXIF USER COMMENT" image.jpg # set EXIF UserComment
You can also extract the EXIF user comments to a CSV with:
exiftool -EXIF:UserComment -csv *.jpg
SourceFile,UserComment
a.jpg,FUNKY EXIF:UserComment
b.jpg,b FUNKY EXIF:UserComment
You can also apply comments from a CSV.
You should also be able to extract macOS/Spotlight/Finder comments using the script in my main answer:
$HOME/macOSGetFinderComment "/Users/someone/soneFile.tif"
Original Answer
I would suggest you try the following using ImageMagick.
First, use the Finder, or any other tool you are familiar with, to make a copy of your photos including the entire directory structure to some new place where we cannot damage your existing photos. So, let's say you copy (NOT move) the entire tree of TIFs to a subdirectory called "NEW" inside your HOME directory.
Then start the Terminal and change directory to "NEW":
cd NEW
Easy Method
If all the TIFs are in a single directory or two, just use mogrify:
mogrify -format jpg *.tif
Harder Method
If the TIF files are in multiple directories, you will need to work a bit harder. Inside Terminal copy and paste this:
find NEW -name \*.tif -exec sh -c 'new="${1%.tif}.jpg"; convert "{}" "$new"' _ {} \;
That starts looking in the "NEW" directory for files named "*.tif". When it finds one, it starts a new shell (sh) passing it the filename of the TIF. It then works out the new filename by replacing a trailing "tif" with "jpg" and invokes ImageMagick convert to do the conversion.
As regards the Finder/Spotlight comments, here is a little script to get the Finder comment of a file:
#!/bin/bash
# macOSGetFinderComment
# Pass an absolute path to the file!
file=$1
osascript<<EOF
tell application "Finder" to get comment of item POSIX file "$file"
EOF
And here is one to set the Finder/Spotlight comment:
#!/bin/bash
# macOSSetFinderComment
# Pass an absolute path to the file!
file=$1
comm=$2
osascript<<EOF
tell application "Finder" to set comment of item POSIX file "$file" to "$comm"
EOF
So, I would save those 2 scripts in your HOME directory and then make them executable with:
cd
chmod +x macOS*FinderComment
Then save this file in your HOME directory under $HOME/CopyComments:
#!/bin/bash
shopt -s nullglob
for f in $(pwd)/*.tif; do
comment=$($HOME/macOSGetFinderComment "$f")
new="${f%.tif}.jpg"
echo Setting comment of $new to $comment
$HOME/macOSSetFinderComment "$new" "$comment"
done
and make it executable with:
chmod +x $HOME/CopyComments
and run it with:
cd NEW
$HOME/CopyComments
I have posted this problem also in Apple Community, here is the solution proposed by VikingOSX. It is a big piece of code, so better download it from here or directly from the Apple Community Link mentioned. Here is a description about the solution as described in the original post:
Prompts for a source folder, and a destination folder.
Duplicates folder hierarchy from source to destination folder.
Selects all TIFF images in the folder hierarchy and converts them to JPEG.
For sub-folders and their files, transfers the original Finder comments, color tags and tag name(s) to the destination hierarchy.
The compression level for the JPG file is high, it can be modified for: medium or low in the line: save this_img as JPEG in outfile_name with compression level medium with icon
Limitation: Source folder can only contain one-level of sub-folders. Ignoring this will result in unplanned results.
Additional Comments
Uses a with timeout clause to allow for large number of files. AppleScript does not yet support Finder tag names, so this script uses AppleScript/Objective-C to get and set those tag name(s). Due to this extension, the script now requires AppleScript 2.4 and must be run on OS 10.10 or later.
Due to the AppleScript/Objective-C code, the script cannot be run interactively as a script/script bundle without using the control+command+R keyboard shortcut. A test is made when the script starts, and will warn appropriately. It is best to save the script as an application to avoid this keyboard shortcut altogether.
Usage
Save the script and then copy and paste the file contains into the Script Editor (you can find the application in the folder: Utilities under the name: Script Editor), compile and save the file with the format: Application, then double click on it to run the script application.
I have tested the script under with Mac Air 2010, with OS El Capitan, for a folder with 884 TIFF files with 2.25GB size and it takes about 18 minutes to convert them into JPG files with medium compression level. The generated files will contain the tags and comments from the original equivalent TIFF file.
Disclaimer
Comment and tags generated in one platform for example Windows or mac OS are not visualized in the other platform. Tags created in Windows are treated in mac OS as keywords (Comand+i for visualizing them), but comments generated in Windows are not visualized in mac OS. This is general incompatibility problem that apply for photos in any format (for example TIFF or JPG).
EDIT (updated solution for solving cross-platform problem with comments)
Taking the idea from #MarkSetchell, I adapted the original script to at least solve the cross-platform problem from macOS to Windows, i.e. a comment from macOS can be seen in Windows platform. The idea is to use EXIF metadata. Then the Applescript will invoke the shell script for invoking the exiftool:
set uxFilepath to POSIX path of NewIMG
do shell script "/usr/local/bin/exiftool -overwrite_original -EXIF:UserComment=\"" & cmtstr & "\" " & uxFilepath
Windows processes the UserComment metadata from EXIF as a regular file comment. Now same comment on the TIF file will be on the JPG and also because such comments were copied (copy-paste) into an EXIF metadata the same information will be visualized under Windows. The same idea can be used for other file properties, in case Windows/Mac read it.
The EXIF metadata in macOS can be visualized from command line as suggest #MarkSetchell, but also from Finder: Command+o (to launch preview app), then Command+i (to launch the inspector). Then click on tap: "More Info", then the tab EXIF.
For the opposite process will require an script that does the opposite, i.e., copy EXIF comment using exiftool, into macOS comment. I have verified that in such case the Windows comment will appear under the label: XPComment. The script uses: UserComment, but it works using XPComment as label in both directions.
I have a script set up to rotate some log files in windows, and as part of the process I'd like it to automatically compress the rotated file. To do this I use the command
compress source.file destination.file.zip
However, if I try to open the file, I get the message "The Compressed (zipped) Folder is invalid or corrupted"
I've tried compress with -Z, and I get the same message. What am I doing wrong?
compress output is not ZIP file format compatible, it uses the LZW algorithm.
The only way to "open" a compressed file is with uncompress or gunzip.
Windows ports of common Unix commands, including compress and gzip/gunzip available here.
EDIT: To produce ZIP files from the command line in Windows, you can use something like 7-Zip, which includes a command line application (7z.exe). The Unix commands linked above also include zip.exe for manipulating ZIP files from the command line.