Montage using PythonMagick in Python 3? - windows

I was hoping to be able to generate montages using PythonMagick. The documentation seems very sparse, but I've been trying to hunt it down using the code completion part of Eclipse at least, as well as a few other questions' suggestions here on Stack Overflow. It seems that the MagickWand API has the function I am looking for, according to this:
http://www.imagemagick.org/api/MagickWand/montage_8c.html
However, I cannot seem to find it in PythonMagick. Is this simply unavailable? If so I might just ditch the rest of my PythonMagick code and rely on subprocess.call on a portable ImageMagick distribution or something like that (this program will have to be portable, and run on Windows with an easy port to Mac OS... so far I have a few other PythonMagick commands working so I'd like to keep this route going if possible).
Thanks!

Using the python imagemagick/graphicsmagick bindings helps a lot, but unfortunately not all of the functionality is there yet. I actually had the same problem with #FizxMike. I needed to use montage and then do some further operations, but saving the file on hard disk and then reloading it in a proper pgmagick object in order to do the rest of the operations and saving it again was slow.
Eventually I used the subprocess solution, but instead of saving in a file, I redirect the output in stdout. Then, I use the stdout to load the image from a pgmagick.Blob in a pgmagick.Image object and do the rest of the processing in python code.
The procedure looks like this in code:
import os
import pgmagick
import subprocess
my_files = []
# Dir with the images that you want to operate on
dir_with_images = "."
for file in os.listdir(dir_with_images):
if file.endswith(".png"):
my_files.append(os.path.join(dir_with_images, file))
montage_cmd = ['gm', 'montage']
montage_cmd.extend(my_files)
# The trick is in the next line of code. Instead of saving in a file, e.g. myimage.png
# the montaged file will just be "printed" in the stdout with 'png:-'
montage_cmd.extend(['-tile', '2x2', '-background', 'none', '-geometry', '+0+0', 'png:-'])
# Use the command line 'gm montage' since there are not python bindings for it :(
p = subprocess.Popen(montage_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
# Get the stdout in a variable
stdout, stderr = p.communicate()
# Load the stdout in a python pgmagick Image object using the pgmagick.Blob
# and do the rest of the editing on python code
img = pgmagick.Image(pgmagick.Blob(stdout))
# Display the image
img.display()
geometry = pgmagick.Geometry(300, 200)
geometry.aspect(True)
# Resize the montaged image to 300x200, but keep the aspect ratio
img.scale(geometry)
# Display it again
img.display()
# And finally save it <- Only once disk access at this point.
img.write('myimage.png')

I have the same problem, even pgmagick lacks the montageImage() function needed (Magick++ montage example)
This is what I do (in a Django View):
#ImageMagick CLI is better documented anyway (-background none preserves transparency)
subprocess.call("montage -border 0 -geometry "+str(cols)+"x -tile 1x"+str(len(pages))+" "+target_path[0:len(target_path)-4]+"[0-9]*.png -background none "+target_path,shell=True)`
Not fun because I have to juggle around a bunch of files first... writing to hard disk is not the fastest thing to do, then delete the temp files.
I would much rather do it all in ram.
I am still in search of a better answer myself.

Related

gitpython: How to show diffs between blobs in a human readable format, as in gitk?

I must be missing something really basic. Given this script:
import git
repo = git.Repo(r'C:/leo.repo/leo-editor')
diff_index = repo.head.commit.diff('HEAD~1')
for d in diff_index:
print('%s %9s %9s %s' % (
d.change_type, id(d.a_blob), id(d.b_blob), d.a_path))
I get something like this:
M 173600704 173600080 leo/core/commit_timestamp.json
M 173600368 173599408 leo/core/leoTest.py
M 173600272 173598928 leo/test/unitTest.leo
So far, so good. This is compatible with what gitk shows, that is, modifications to the three files shown.
But now, having access to the a_blob and b_blob objects for each file, how do I get a human-readable diff of the differences between those two blobs? In other words, I want to recreate what gitk shows.
I don't see anything in the docs related to this.
Edward
My question is a variant of this stack-overflow question.
given a blob, blob.data_stream.read() returns its raw contents, that is a <str> object on Python 2, and a <bytes> object on Python 3.
Rather than reading the feeble api docs for Objects.Blob, one would be better off reading the source code. Indeed, Objects.Blob is a subclass of base.IndexObject, which in turn inherits the data_stream property from base.Object (not to be confused with object).

Editing thickness in postscript (.ps or .eps) figures via unix shell commands?

I have many figures (graphs) in postscript (.eps) format that I wish to thicken the plots with.
I found the following code, but the output file is no different. I was wondering what I was doing wrong.
The code:
# get list of all arguments
set args = ($*)
# if not enough arguments, complain.
if ($#args < 2) then
echo "Usage: ps_thicken ps_file factor"
echo "Thickens all lines in a PostScript file by changing the linewidth macro."
echo "Result goes to standard output."
exit 1
endif
sed -e "s/^\/lw {\(.*\) div setlinewidth/\/lw {$2 mul \1 div setlinewidth/" $1
Now to execute this from my command line, I use the command (filename is ps_thicken, and has appropriate permissions):
./ps_thicken old_file.eps 10 > new_thick_file.eps
Which I thought should make everything 10x thicker, but it just doesnt change anything.
Any help would be greatly appreciated, I'm pretty new to shell script!
PostScript is a programming language, so it isn't really possible to make changes in an automated fashion like this. At least not without writing a PostScript program to do so!
Note that linewidth isn't a 'macro' (PostScript doesn't have macros) its am operator. What the code you've posted for sed does (if I recall sed well enough) is look for the definition of /lw and replace it with a modified version. The problem with that is that /lw is a function declartation in a particular PostScript program. Most PostScript programs won't have (or use) a function called 'lw'.
You would be much better to prepend the PostScript program code with something like:
/oldsetlinewidth /linewidth load def
/setlinewidth {2 div oldsetlinewidth} bind def
That will define (in the current dictionary) a function called 'setlinewidth'. Now, if the following program simply uses the current definition of setlinewdith when creating its own functions, it will use the redefined one above. Which will have the effect of dividing all line widths by 2 in this case. Obviously to increase the width you would use something like 2 mul instead of 2 div.
Note that this is by no means foolproof, its entirely possible for a PostScript program to explicitly load the definition of setlinewidth from systemdict, and you can't replace that (at least not easily) because systemdict is read-only.
However its unlikely that an EPS program would pull such tricks, so that should probably work well enough for you.
[based on comments]
Hmm, you mean 'failed to import' into an application or something else ?
If you're loading the EPS into an application then simply putting that code in front of it will break it. EPS (unlike PostScript) is required to follow some rules, so to modify it successfully you will have to follow them. This includes skipping over any EPS preview.
This is not really a trivial exercise. Your best bet is probably to run the files through Ghostscript, you can do a lot by harnessing a PostScript interpreter to do the work.
Start with the 2 lines of PostScript above in a file, then run the EPS file you want to 'modify' through Ghostscript, using the eps2write device. That will produce a new EPS which has the changes 'baked in'.
Eg (assuming the linewidth modifying code is in 'lw.ps'):
gs -sDEVICE=eps2write -o out.eps lw.ps file.eps
But be aware that the resulting EPS is a completely rewritten program and will bear no relation to the original. In particular any preview thumbnail will be lost.

Creating animation with multiple plots in Octave

I'm using Octave to write a script that plots a function at different time periods. I was hoping to create an animation of the plots in order to see the changes through time.
Is there a way to save each plot for each time point, so that all plots can be combined to create this animation?
It's a bit of kludge, but you can do the following (works here with octave 4.0.0-rc2):
x = (-5:.1:5);
for p = 1:5
plot (x, x.^p)
print animation.pdf -append
endfor
im = imread ("animation.pdf", "Index", "all");
imwrite (im, "animation.gif", "DelayTime", .5)
Basically, print all your plots into a pdf, one per page. Then read the pdf's as images and print them back as gifs. This will not work on Matlab (its imread implementation can't handle pdf).
This creates an animated gif
data=rand(100,100,20); %100 by 100 and 20 frames
%data go from 0 to 1, so lets convert to 8 bit unsigned integers for saving
data=data*2^8;
data=uint8(data);
%Write the first frame to a file named animGif.gif
imwrite(data(:,:,1),'/tmp/animGif.gif','gif','writemode','overwrite',...
'LoopCount',inf,'DelayTime',0);
%Loop through and write the rest of the frames
for ii=2:size(data,3)
imwrite(data(:,:,ii),'/tmp/animGif.gif','gif','writemode','append','DelayTime',0)
end
Had to come chime in here because this was the top Google result for me when I was looking for help with this. I had issues with both answers, and some other issues, too. Notably:
For Rick T's answer, the code snippet doesn't write a plot figure, it just writes matrix data. Getting the plot window was a pain.
For carandraug's answer, writing to a PDF took a very long time and made a gigantic PDF.
On my own machine, I'm pretty sure I used apt-install to get Octave, but the getframe function I found referenced in other answers wasn't found. Turns out I had installed version 4.4, which was from 2018 (>3 years old).
I removed the old version of Octave sudo apt remove octave, then installed the new version with snap. If you try octave from a terminal without it installed it should prompt you to the snap install - be sure to include the # 6.4.0 or whatever is included in the command.
Once I had the current version installed, I got access to the getframe command, which is what lets you convert directly from a figure handle to image data - this bypasses the hackey (but previously necessary step) in #carandraug's answer where you had to write to PDF or some other image as a placeholder.
I used #RickT's answer to make my own MakeGif function, which I will share with you all here. Note that MakeGif stores the filename in a persistent variable, meaning it is retained across calls. If you change the filename it will make (or overwrite!!) the new file. If you need to overwrite the current file (i.e., running the same script multiple times and want new results) then you can use clear MakeGif between calls and that will reset the persistentFilename.
Here is the code for the MakeGif function; code to test it with is provided after this:
function MakeGif(figHandle, filename)
persistent persistentFilename = [];
if isempty(filename)
error('Can''t have an empty filename!');
endif
if ~ishandle(figHandle)
error('Call MakeGif(figHandle, filename); no valid figHandle was passed!');
endif
writeMode = 'Append';
if isempty(persistentFilename)|(filename!=persistentFilename)
persistentFilename = filename;
writeMode = 'Overwrite';
endif
imstruct = getframe(figHandle);
imwrite(imstruct.cdata, filename, 'gif', 'WriteMode',writeMode,'DelayTime',0);
endfunction
And here is the code to test the function. There's a commented-out call to clear MakeGif between the blue and green colors. If you leave it commented out it will append the green sine wave to the blue sine wave, resulting in alternating colors after every cycle - again the filename is persistent in the function. If you uncomment that call then the MakeGif function will treat the green's call as "new" and trigger the overwrite of the blue sine wave and all you'll see is green.
clear all;
time = 0:0.1:2*pi;
nSamples = numel(time);
figHandle = figure(1);
for i=1:nSamples
plot(time,sin(time + time(i)),'Color','blue');
drawnow;
MakeGif(figHandle, 'test.gif');
endfor
% Uncomment the 'clear' command below to clear the MakeGif persistent
% memory, which will trigger the green sine wave to overwrite the blue.
% Default behavior is to APPEND a green sine wave because the filename
% is the same.
%clear MakeGif;
for i=1:nSamples
plot(time,sin(time + time(i)),'Color','green');
drawnow;
MakeGif(figHandle, 'test.gif');
endfor
I spent several hours on this after being super dissatisfied with laggy screen captures so I really hope this helps someone in the future! Good luck and best wishes from the Age of Covid lol.
#Chuck thanks for that code; I've been using it to save 1500-frame GIFs of simulation output, and I find that after maybe ~500 frames the time to save the next frame to the output during the call to MakeGif starts to become ... unnerving. I guess imwrite reads and writes the entirety of the output file at each call that includes the 'WriteMode','Append' pair. At frame 1500 my output is 480Mb so that becomes untenable.
An apparent rescue for this is hinted at in the doc for Octave 7.1.0's imwrite, with the suggestion that you can pass it a 4-dimensional array and write the entire image sequence with one call. I haven't been able to make this work, though: Calling imwrite that way seems to simply write the very first image in the sequence into every frame in the output file.

Convert PDF files to PDF/A via Ghostscript

I'd like to convert arbitrary PDF files to PDF/A with Ghostscript 9.15.
Is Ghostscript able to create PDF/A-3b conformant PDFs? There is no parameter which represents a PDF/A conformance level, so I assume there is no possibility. Or is there anything I have overlooked?
I was following a blog post where a Windows batch file is used to convert from PDF to PDF/A (see http://www.mcbsys.com/techblog/2013/04/batch-convert-pdf-to-pdfa/). The gs invokation in the batch is:
"%gs_path%\gswin64c" ^
-dPDFA ^
-dNOOUTERSAVE ^
-sProcessColorModel=DeviceRGB ^
-sDEVICE=pdfwrite ^
-o "GS_%file1%" ^
-dPDFACompatibilityPolicy=1 ^
"%currentdir%\PDFA_def.ps" ^
%inputfilelist%
The PDFA_def.ps is an adjusted version of the official one:
%!
% This prefix file for creating a PDF/A document is derived from
% the sample included with Ghostscript 9.07, released under the
% GNU Affero General Public License.
% Modified 4/15/2013 by MCB Systems.
% Feel free to modify entries marked with "Customize".
% This assumes an ICC profile to reside in the file (AdobeRGB1998.icc),
% unless the user modifies the corresponding line below.
% The color space described by the ICC profile must correspond to the
% ProcessColorModel specified when using this prefix file (GRAY with
% DeviceGray, RGB with DeviceRGB, and CMYK with DeviceCMYK).
% Define entries in the document Info dictionary :
/ICCProfile (... PATH TO ... AdobeRGB1998.icc) % Customize.
def
[ /Title (Title) % Customize.
/DOCINFO pdfmark
% Define an ICC profile :
[/_objdef {icc_PDFA} /type /stream /OBJ pdfmark
[{icc_PDFA} <</N systemdict /ProcessColorModel get /DeviceGray eq {1} {systemdict /ProcessColorModel get /DeviceRGB eq {3} {4} ifelse} ifelse >> /PUT pdfmark
[{icc_PDFA} ICCProfile (r) file /PUT pdfmark
% Define the output intent dictionary :
[/_objdef {OutputIntent_PDFA} /type /dict /OBJ pdfmark
[{OutputIntent_PDFA} <<
/Type /OutputIntent % Must be so (the standard requires).
/S /GTS_PDFA1 % Must be so (the standard requires).
/DestOutputProfile {icc_PDFA} % Must be so (see above).
/OutputConditionIdentifier (AdobeRGB1998) % Customize
>> /PUT pdfmark
[{Catalog} <</OutputIntents [ {OutputIntent_PDFA} ]>> /PUT pdfmark
So, I use AdobeRGB1998.icc which is obviously useable for PDF files with RGB color space. Depending on the -sProcessColorModel value (DEVICERGB) a correct value is printed out.
The conversion works for all files. But when I validate the created PDF file against PDF/A-1b, I get different results depending whether the input file has RGB color space or not (e.g. CMYK). So, when I have an input PDF file which uses CMYK color space, the file gets converted by the script, but the validator says something like this:
input.pdf", 1, 38, 0x03418614, "A device-specific color space (DeviceCMYK) without an appropriate output intent is used.", 1
"output.pdf", 20, 0, 0x83410612, "The document does not conform to the requested standard.", 1
My question: Is there a way to get the conversion done for arbitrary files (i.e. independent of the used color space in the input file)?
Update
#KenS Thanks for your answer. I've updated my initial post to clarify what I want to achieve.
To make it more explicit, I will use an example. There are two files: input1.pdf (seems to use RGB) and input2.pdf (seems to use CMYK). I want to convert both of them to PDF/A-1. Thanks to your hint, I've let go of the above mentioned batch script and instead tested the command directly in the command line. After reading Ps2pdf.htm#PDFA, I have adjusted the (official) PDFA_def.ps so that AdobeRGB1998.icc is used. Then I invoked the following command on both input files (replaced output1.pdf by output2.pdf and input1.pdf by input2.pdf for the second file):
gswin64c.exe -dPDFA=1 -dBATCH -dNOPAUSE -dNOOUTERSAVE \
-sColorConversionStrategy=/RGB \
-sOutputICCProfile=AdobeRGB1998.icc -sDEVICE=pdfwrite \
-sOutputFile=output1.pdf -dPDFACompatibilityPolicy=1 \
"PATH/TO/OFFICIAL/PDFA_def.ps" input1.pdf
The conversion was done without any errors. The output1.pdf seems to be valid, but the output2.pdf is still invalid (tested with 3heights Validator):
"output2.pdf", 1, 40, 0x03418614, "A device-specific color space (DeviceCMYK) without an appropriate output intent is used.", 1
"output2.pdf", 20, 0, 0x83410612, "The document does not conform to the requested standard.", 1
So when I understand your answer correctly, the above command should produce a pdf file which uses the RGB color space - independent of the color space of the input file. If the input file uses CMYK, than the colors have to be translated into RGB with the above command.
When I interpret the first error message correctly, the used color space in the output2.pdf is still CMYK (although the command parameters like ColorConversionStrategy=/RGB). Since I used AdobeRGB1998.icc, the validation error appears.
What am I missing in the above command?
Going back to my original question (which is one step further): Instead of always converting to RGB (or CMYK), I wanted to somehow detect which color space is used in the input file and then dynamically switch to a RGB or CMYK icc file. Is it possible to achieve that?
Ghostscript does not support PDF/A-3. The conformance parameter you are looking for is -dPDFA= where valid values are nothing (defaults to 1), 1 or 2. You can find this documented in ghostpdl/gs/doc/ps2pdf/htm#PDFA
I'm not sure what you are asking for here though. You must either create a PDF/A file (in level 1 or 2 anyway, I haven't read the revision 3 spec yet) which is RGB or CMYK, because you aren't allowed to use both (you can convert everything to device independent colour of course). The colour space used in the input isn't relevant, other than to decide whether it needs to be converted.
This is something you need to decide, we can't decide it for you. One important reason is that the OutputIntent must be consistent with either RGB or CMYK, and the pdfwrite device doesn't check it, it assumes you chose one which matches the device space you are using for the PDF file (by the way, don't set the ProcessColorModel, use ColorConversionStrategy instead) In your case you have set OutputIntent to AdobeRGB1988 so your colours must be specified either in device independent colour, or RGB.
Given the errors you quote, I would suggest the problem is that you haven't specified -sColorConversionStrategy, so the input colours are not being converted to the required device space. I would further guess that the script you copied this from set -dUseCIEColor, and you didn't copy that bit. DO NOT set -dUseCIEColor, its a horrbile ancient piece of PostScript hackery. Instead set ColorConversionStrategy, which will convert colours in a much better way, as required.
Updated answer as this started getting too long for a comment:
I can't immediately see any problems with your command line, can you share an example PDF file ? Its much easier to investigate these things with a solid example. I know from our customers and other free users that pdfwrite is capable of producing conforming PDF/A-1b files.
Regarding the second question; its not possible to do that because currently you need to set the OutputIntentProfile to either a CMYK one or an RGB one before you start. You can't just run through the input PDF file until you come to a colour operation and then decide. If you feel like some programming it could be done by modifying pdfwrite, because the profile isn't actually used till the output is closed.
One problem is that, in order to do the colour conversion, you need to set the underlying ProcessColorModel (this is done for you automatically by ColorConversionStategy). The only way to change ProcessColorModel is to execute a setpagedevice, which causes an erasepage. Now I think that's actually fixable with pdfwrite, all it does is write a white rectangle over the page, so you should be able to intercept that and not emit it. Otherwise any marks you made before you encountered an RGB or CMYK operation would be underneath the white rectangle.....
So essentially no, you can't do it right now, if its important to you then you could probably modify the code to do so (don't forget you will also need to supply 2 OutputIntent profiles to choose between as well). We've never had a customer request to do this, so we won't likely take it on as a project. Of course if you did get this working we might very well incorporate it into the code base if you were to offer it back to us.

Convert a .rtf into a mac .r resource, in a scriptable way

I currently have a SLA in a .rtf format, which is to be integrated into .dmg using the intermediary .r mac resource format, which is used by the Rez utility. I had already done it by hand once, but updates made to the .rtf file are overwhelming to propagate to the disk image, and error-prone. I would like to automate this task, which could also help adding other languages or variants.
How could the process of .rtf to .r text conversion be automated?
Thanks.
Only because I didn't fully understand how the accepted answer actually achieved the goal, I use a combination of a script to generate the hex encoding:
#!/usr/bin/env ruby
# Makes resource (.r) text from binaries.
def usage
puts "usage: #{$0} infile"
puts ""
puts " infile The file to convert (the output will go to stdout)"
exit 1
end
infile = ARGV[0] || usage
data = File.read(infile)
data.bytes.each_slice(16) do |slice|
hex = slice.each_slice(2).map { |pair| pair.pack('C*').unpack('H*')[0] }.join(' ')
# We could put the comments in too, but it probably isn't a big deal.
puts "\t$\"#{hex}\""
end
The output of this is inserted into a variable during the build and then the variable ends up in a template (we're using Ant to do this, but the specifics aren't particularly interesting):
data 'RTF ' (5000, "English SLA") {
#english.licence#
};
The one bit of this which did take quite a while to figure out is that 'RTF ' can be used for the resource directly. The Apple docs say to separately insert 'TEXT' (with just the plain text) and 'styl' (with just the style). There are tools to do this of course, but it was one more tool to run and I could never figure out how to make hyperlinks work in the resulting DMG. With 'RTF ', hyperlinks just work.
Hoping that this saves someone time in the future.
Use the unrtf port (from macports), then format the lines, heading and tail with a shell script.

Resources