Which tag is used to compare images using computer vision in PAF?
For example, I want to check whether an image is present in the full screen of web application while automation. The tag should be able to return the precision.
CV tag can be used to compare images using computer vision API. This is supposed to help us automate static UI testing. The way it would work is as follows.
Screenshot of the screen area to be validated would be captured while writing scripts.
Screenshot of the full screen will be taken at runtime.
The screen area captured at the time of script creation would be compared against the screenshot taken at runtime.
The output would be a screenshot where the matched image would be marked in the screenshot, x and y coordinates (in variables) and the matching precision (in variable).
Generally, when the image matches, the precision is above 0.995 or something. When the script is being built, we need to check the precision number and validate the same with validation tag.
Syntax:
<cv screenshotFile="" fileToMatch="" outputFile="" resultVarName="" locXVarName="" locYVarName="" desktop=""></cv>
screenshotFile - contains the location where the screenshot will be
saved at runtime.
fileToMatch - contains the path of the screen area image that
needs to be checked in the screenshot
outputFile - path where the output file containing the image
matching result needs to be saved
resultVarName - name of the variable which will contain the
matching precision output
locXVarName - Name of the variable which will contain the x
coordinate where the image has matched in the screenshot
locYVarName - Name of the variable which will contain the y
coordinate where the image has matched in the screenshot
desktop -
whether the screenshot is a robot screenshot or not. If this is
false, it will use selenium to take the screenshot in the browser.
If true, it will use robot screenshot.
Related
Noticed that images sometimes are sliced up in PDFs.
Steps:
insert an image with a high resoultion (3000x1800) into a .docx
use "Microsoft Print to PDF" option of Word to convert to PDF
extracting all images with pdfimages or pymupdf
Result:
Image is sliced horizontally into three images
Questions:
What exactly happens in the in the transition from .docx to pdf (or in generell in the process to pdf) that makes the converter slice it up into three images instead of one?
Do the individuell XObjects of the sliced images contain information which says that these three images belong to originally one?
How do I know how the images are sliced (horizontally / vertically) and what if originally there were two images inserted into the .docx file and both of them are sliced. Can you tell if slice x belongs to original image y or z?
So, as you have found out: because the code which generates the PDF choose to do so.
The technical reasons may be various - it could be that historically there were printers which would only have so much memory, and would need to get limiterd size-images when printing, and someone at some point when writing the PDF export code present in Microsoft Office choose to apply this limit.
Anyway, technically, as put in the comments, an image in a PDF file could be composed of unlimited smaller images collated together.
Now, the second part, and your actual question: to know whether images ibn a PDF file belong together in a single original image one would need a custom extractor tool to check the geometry of all images in the document and find out which images have no margins or boundaries with others - it would not be that hard to do for well behaved files (which we can't know if MS Office generated files are: there are ways to obfuscate image positioning by making it indirectly). The metadata in the image-parts may or may not contain information that would allow one to recompose the original image: it would be up to the code generating the PDF to include this metadata or not - but the geometry can't lie in this case: if the final document presents a single image visually, it is possible to detect that when fetching the images.
I am trying to print the images from the ioComp iPlotX object to a PDF printer. This object has iPlotX.SaveToBitmap, which does work and I can 'send' the file to the PDF printer, but the image is blurry, the bitmap is 96 DPI.
The control has iPlotX.PrintChart method which is supposed to go directly to a printer. There is also a method to indicate which Windows printer to direct the output to. I fumbled around to see if I could get anything to work and a couple times I produced an image in the PDF file that was very clear. But other items printed to the page were gone and I only had one page instead of 3 pages. Most of the times nothing is in the PDF file.
A snippet of my relevant code is below.
Me.SinglePlotObject1(iIndex).PlotControl.PrintShowDialog = False
Me.SinglePlotObject1(iIndex).PlotControl.Printername = modNovaPDFConstants.NOVAPDF_PRINTER_NAME
Me.SinglePlotObject1(iIndex).PlotControl.PrintOrientation = poLandscape
Me.SinglePlotObject1(iIndex).PlotControl.PrintChart
This note is from the ioComp.Help:
Note : If you have set PrintShowDialog to TRUE and have set
PrinterName to a non-empty string, then the printer selected by the
user will be ignored! The PrinterName property will always override
the user printer selection, if you are using our printer dialog If
you plan on showing our built-in printer selection dialog to your end
user, then you should set the PrinterName property to an empty string
for everything to work correctly.
Also from their help is :
property PrintXStart : Double Used to specify the starting x-axis Min
value when implementing multi-page printing.
Description: Use PrintXStart to get or set the starting x-axis Min
value when implementing multi-page printing. By default, the
PrintXStart and PrintXStop are initialized to the Min and Max of the
x-axis display on the screen and will only print out one page. You
must manipulate the PrintXStart and PrintXStop in the OnBeforePrint or
OnBeforePrintPage event to have multiple pages print out during a
single print session.
The only example they show is setting the XStart property to 0. I am not sure if this means multiple pages for the same graph, that is, the graph is stretched out over multiple page or is this has something to with different iPlots on separate pages. In the past we have always just used the iPlotX.SaveToBitmap to print. At that time the plots and text were good enough.
If anyone has any experience with ioComp plot controls and printing, I am willing to try any suggestions.
Right now I am trying to create a test application with as little code as possible to test all of their other printer options.
I have tried their FAQ section but could not find anything related to printing using VB6 and the ActiveX controls. When I sent a message to them, a smtp error appeared.
DaveInCaz: Good Idea. I tried a different PDF printer and results are the same. I tried a method of the control called iPrintToDC, passing in the Printer.hDC. This made some progress. I can print two graphs on two pages, but the graphs are about the size of my thumb. If I change width's or height's, it only makes things worse, by somehow placing the bottom of the graph 'above' the title and removing all the contents of the graph itself; it is as if the graph is wrapped around a cylinder and all I see is where the top and bottom meet. When I used the .PrintChart method, only the last of two charts would be printed and nothing else I printed would appear.
Thanks.
I have a list of images that I am currently looping through.
Is it possible to have a command where I can put in the location of text that I am trying to extract or show it with something like Citrix Scrape on what data I want to take out?
The Get OCR Text action requires you to provide a UI element. One simple option is to loop over all images, then open them in an image viewer of your choice, and then use the Text -> Scrape Relative to extract your invoice number into a variable.
You can use the Desktop Recorder for all actions mentioned above. Note that you will have to adjust the Selectors, as the Recorder usually considers the text the application title as well (which will often include the document's name, and that's not what you want as all your invoices will have different names). You can just open another invoice in your application, then open the Selector and Attach to Live Element to let UiPath work out the Selector for you - which works just fine in most cases.
Depicted below: a sample sequence created with the Desktop Recorder. Here I just read the invoice number with OCR, and I use IrfanView to display them. Of course you may want to increase the zoom level to enhance your OCR results in a real-world scenario.
i want to generate two page report with different background image (the image is static btw).
The first one would be use for details and the second one would be just a plain image
Do you want one image to be on page one and the other on page two? If so, in the properties menu for the image, find the Print When Expression element and use the $V{PAGE_NUMBER}.equals() expression.
I need to develop a desktop application which will
1.) have a list of the Different Application logos (Background Transparent) e.g. IE, FIREFOX, CHROME, PHOTOSHOP ETC.
2.) User will take a screenshot of desktop and save the image.
3.) Now my application need to search all the logos in the screenshot image and tell which all logos are present and where.
4.) I used OPENCV, it's working, but when user changes the desktop background & captures screenshot, it's not working as the transparent area of logo is getting the desktop background content.
Can somebody provide a solution or libraries open source, commercial to do this job.
This is easy to do using cross-correlation.
See my answer to this question.
Basically:
Start with desktop image and one template image for each icon
Apply edge detection (e.g. Sobel) to the desktop image and template images.
Throw away the original desktop image and templates, you won't need them anymore cause we'll be using the edge-detected images
For each template
Do template matching as you normally would
Threshold the maximum of the result. If it's above the threshold, you have a match at that position. Otherwise, no match.
If your icons are aligned in a grid on the desktop, you may be able to speed up your processing by only checking those specific grid positions.
EDIT
You can also save a lot of time by knowing which icons to search for. If you have access to the file system, then just look for *.lnk files (or any other extensions you may be interested in) in the directory that corresponds to the desktop (can't remember exactly what it is, but for Windows7 it's something like c:\users\misha\desktop). That will tell you what icons are there on the desktop. This will allow you to shorten your template candidate list before you go and do the template matching.
I like misha's answer and I think it should work for you. But it that doesn't work you could try replacing the transparant pixels in your reference logo with uniformly distributed random noise before trying the match. This will make the transparant pixels irrelevant for any matching computation because they will match just as bad no matter what there is on the desktop in those pixels.
I'm not familiar with the tools you're using, but I'm guessing you have to either:
a) Tell your program to ignore transparent pixels in the icon images during the comparison operation.
OR
b) Tell your program to treat transparent pixels in the icon images as "wildcards" which can be any color.