Try as I might..
I can't get the displayed text using pywinauto from an in house application tool (Eurotherm's iTools), written with Delphi. I have written a large amount of Python test code (infrustructure) using pywinauto. I plotted the browser x,y elements using a text file for the expanded and indented structure coordinates because there is no text within the browser 'TUivOPCTreeView' frame.
I managed to select TtsGridEditBox via DoubleClickInput and highlight it with DrawOutline(). All efforts .UserData(), .Texts(), .WindowText(), looping through children filtering lamda's and a multitude of crashing commands give text of ''. I think my only option is to copy the text as it seems it can't be read.
I managed to delete (.TypeKeys("{HOME}{DELETE}{DELETE}{DELETE}{DELETE}{DELETE}") and enter a different value (.TypeKeys(' + var + '"{ENTER}")).
Is there a way to issue: home, select, end, copy with .TypeKeys or anything else?
Control Identifiers:
TtsGridEditBox - '' (L680, T498, R776, B512)
'49' 'TtsGridEditBox' ()
Read WindowText from valueNum20 is {u'UserData': 0, u'Style': 1409286144, u'ControlCount':
0, u'ClientRects': [<RECT L0, T0, R96, B14>], u'IsEnabled': True, u'Fonts': [<LOGFONTW 'M
S Shell Dlg' -11>], u'FriendlyClassName': u'TtsGridEditBox', u'IsUnicode': True, u'Texts':
[''], u'ContextHelpID': 0, u'ExStyle': 0, u'IsVisible': True, u'ControlID': 2100226, u'Me
nuItems': [], u'Class': u'TtsGridEditBox', u'Rectangle': <RECT L680, T498, R776, B512>}
The GUI is written in Delphi XE2 (2012) v16.0.4504.48754 with plugins
TeeChart 2012, VirtualTreeViewV5.1.3 and TopGrid 2_20.
Has pywinauto ever been run on non win32 written apps, such as Delphi?
Does this ring any bells with anyone?
I'm convinced that the Delphi written GUI displays but strangely masks the text for any automated tools.
Much, frustrated, thanks in advance :)
Could find an implementation of UI Automation patterns for some Delphi controls including StringGrid.
This parent project "DelphiUIAutomation" can help you using pywinauto with these enhanced controls. Or you may choose Delphi itself for your test code.
Related
I'm trying to handle background color properly in a dynamically generated property sheet in dynamically generated property pages in win32 api using MFC (though I expect my question is general, and not restricted to MFC, but since my code and examples use it, it's germane to my question anyway).
So we have a:
CPropertySheet
containing multiple
CPropertyPage
I generate the contents of any given page dynamically - from file resources using a custom dialog definition language - all irrelevant other than to say - a list of controls and their coordinates is created within a given page, and the page is resized to accommodate them. This logic is working beautifully.
However, what doesn't work is that the controls and background of each page draws using the dialog default color/brush.
I've tried a number of ways to attempt to force it to draw using the white color/brush that a hard-coded property sheet / page would.
There are two important pieces to this:
Page Background
Control (on the page) background
For #1, I've tried:
acquire the background brush from parent window class (it's dialog bkgrd) (same is true if I do this and ask the tab control)
change the property page to use WS_EX_TRANSPARENT (PreCreateWindow is not called by the framework when generating a page viz PropertySheet::AddPage)
For #2, I've tried:
overriding OnWndMsg / WM_CTLCOLORSTATIC to forward that request to (A) the parent (sheet), and (B) to the tab control (which is what wants the white in the first place).
However, anytime I use any of the above "ask for the background / forward the request" up the chain to either the sheet or the tab control - I get the dialog background color, never the white I'd expect.
Using Spy64, I can see that for a fully hardcoded property sheet / page - that the only discernable differences I can see is that the dialog window created in AddPage (or its moral equivalent) has WS_CHILD instead of mine which has WS_POPUP (the rest of the styles appear to be the same, such as WS_VISIBLE|DS_3DLOOK|DS_FIXEDSYS|DS_SETFONT|DS_CONTROL and WS_EX_CONTROLPARENT.
So, other than the WS_CHILD, I see no significant differences from what I'm creating and from another property sheet that works properly from a standard resource (i.e. hard coded).
I'm also flummoxed as to how this works normally anyway - since forwarding things like the ctrlcolor message doesn't respond correctly - and asking for the windows background colors similar doesn't - then how is it in a standard case the background colors of controls and pages comes out as white, and not dialog background?
Any ideas or help would be appreciated - I'm kind of running out of ideas...
When Visual Styles were added in Windows XP they really wanted to show off this new feature so they made the tab background a gradient (really a stretched image) instead of a single color and this caused problems in old applications that did custom drawing with the dialog brush as the background.
Because of this, only applications with a comctl32 v6 manifest got the new look but there was a problem; old propertysheet shell extensions would load in new applications (including Explorer) and things would look wrong.
To work around this they also require you (or your UI framework) to call EnableThemeDialogTexture(.., ETDT_ENABLETAB) to get the correct tab page look.
As if things are not tricky enough, there is a undocumented requirement that you also need a button or a static control on the page!
If you have custom controls they have to call DrawThemeParentBackground when you draw if they are partially transparent.
Turns out my old code had defined an ON_WM_ERASEBKGND handler - and removing that (and all of my above attempts) makes it work.
So simply doing NOTHING is the correct answer. D'oh!!!
I'm leaving my shame here in case someone else trips on this! [Whoops!]
(Still interested if anyone has deeper insight into how this mechanism works under the hood)
I have an old windows application with it's Ocxs. I want to localize it's OCX to arabic. no problem on changing labels and strings.
but I can't change layout to Right to left.
I find some resources about using Mirroring in windows. but the provided samples don't help me. Link1 & Link2
I'm not a VB fan and don't have enough experience.
Is there any clear and tested approach for VB to mirroring UI?
From Platform SDK 2001
Complex Scripts in Edit Controls
A complex script is a language whose printed form is not laid out in a simple way. For example, a complex script may allow bi-directional rendering, contextual shaping of glyphs, or combining characters. The standard edit controls have been extended to support multilingual text and complex scripts. This includes not only input and display, but also correct cursor movement over character clusters (in Thai and Devanagari script, for example).
A well-written application will receive this support automatically, without modification. Again, you should consider adding support for right-to-left reading order and right alignment. In this case, toggle the extended style flags of the edit control window to control these attributes, as shown in the following example:
// ID_EDITCONTROL is the control ID in the resource file.
HANDLE hWndEdit = GetDlgItem(hDlg, ID_EDITCONTROL);
LONG lAlign = GetWindowLong(hWndEdit, GWL_EXSTYLE) ;
// To toggle alignment
lAlign ^= WS_EX_RIGHT ;
// To toggle reading order
lAlign ^= WS_EX_RTLREADING ;
After setting the lAlign value, enable the new display by setting the extended style of the edit control window as follows:
// This assumes your edit control is in a dialog box. If not,
// get the edit control handle from another source.
SetWindowLong(hWndEdit, GWL_EXSTYLE, lAlign);
InvalidateRect(hWndEdit, NULL, FALSE);
Windows 2000/XP: The standard edit control supports a context menu that allows the user to toggle the reading order and insert/display Unicode bi-directional control characters.
Babylon dictionary and a couple of other dictionaries allow to click on any word in any windows program
and automatically recognize the word under the cursor, and at once open the dictionary window while searching for that word in installed dictionaries.
You can on the other hand open your dictionary, type in your word and press Enter, the result will be the same.
There's a Delphi form, containing a text label, for example with the word "Automaton".
My question is:
How to send a word from my Delphi application right into the dictionary window, as if you typed it manually and pressed Enter?
The best solution is to send some message through the Windows mechanism, but if it is too complicated, there's another solution, and so the second answer: as I described, we need to model a [Ctrl+left mouse] click on a form where this word is displayed on a form [ a visual label on the screen of my Delphi application], to be exact, on some central pixel of this label.
Could you kindly give an advice how to do one thing or another in Delphi ?
** edit:
The problem with AppActivate is this: Babylon dict has a daemon part that seats in the tray.
In the task manager a real window where the text should be input also is named 'Babylon'.
So AppActivate('Babylon') tries to bring to front the non-visual part of the application.
Do you have any suggestion how to determine the windows handle or something of a real visual part of the application? In the task manager, I repeat both visual and non-visual parts are named 'Babylon'.
I cannot offer an answer so much as some insight and advice...
There are certain applications which "intercept" keyboard and mouse instructions, and essentially "nullify" them if they are being immitated by software. Generally-speaking, you'd only see this in proper AntiVirus software such as Kaspersky by design... however:
The way some (not many, but some) programs hook keyboard and mouse inputs, as a side-effect, behave the same way. If you have attempted all of the advice given as comments above, and cannot get Babylon to trigger an action as a result, it is likely Babylon behaves as I have described.
If what I suspect is true, then the method you are attempting is simply not possible (at least, not using any simple Pascal code on its own... ASM might be able to do it but that's beyond my knowledge).
A better solution may be to do a little research to see if any of the following options are available to you:
1) Does Babylon have a Pipeline or API you can use to interface your application(s) with it?
2) Is the particular functionality you require of Babylon accessible through one (or more) DLL files distributed as part of Babylon?
3) Is there an alternative to using Babylon for your needs?
I know it's not an answer as such (certainly not one you'd want to hear), but it may point you in a better direction.
Hi i want to make a sort of a bot for WLM,
But the normal controlsend in autoit wont work
the basic question is how can i send a text to the windows live messenger window Without activating it?
Example:
There is someone in WLM with the window active called "Joop" I do:
ControlClick("Joop","","[CLASS:DirectUIHWND; INSTANCE:2]","Left",1,322,507)
ControlSend("Joop","","[CLASS:DirectUIHWND; INSTANCE:2]","Hi Joop")
this doesn't work however when I do this
WinActivate("Joop","")
ControlClick("Joop","","[CLASS:DirectUIHWND;INSTANCE:2]","Left",1,322,507)
ControlSend("Joop","","[CLASS:DirectUIHWND; INSTANCE:2]","Hi Joop")
it does work however now its activated what i dont want.
there is a problem with the window the whole msn talk window is one window so no extra control to type in thats why i tried the contolClick on the coordinates of the typing screen without succ6
this is the summary.
Text:
Position: 0, 28
Size: 882, 607
ControlClick Coords: 282, 180
Style: 0x56000000
ExStyle: 0x00010000
Handle: 0x00190916
>>>> Mouse <<<<
Position: 1885, 557
Cursor ID: 0
Color: 0xFFFFFF
>>>> StatusBar <<<<
>>>> Visible Text <<<<
>>>> Hidden Text <<<<
CVoiceVideoAvatarHostWindow
does anyone know how to get something in that textfield without activating it (so it can run on background) and using a lame function like send.
OR how can i send a text to the windows live messenger window Without activating it
Thanks
Matthy
MSN is purposefully resisting automation. It's not a limitation of the Windows API. If your goal is to automate a MSN chat, you can do so either by directly sending the TCP packets, or by automating another application which supports MSN (Digsby, Pidgin, etc.) and which works better with automation.
By directly sending the TCP packets you are writing your own program, from scratch, which to the outside world behaves just as MSN. There are user-made documentation about the protocol. One that I have used in the past is here: http://www.hypothetic.org/docs/msn/notification/authentication.php but I am entirely unsure whether it is still up-to-date. If you are familiar with networking, TCP, packet capturing and replaying, then I do suggest going this route as it's the most stable, long-term approach. (And I think: The most fun.)
However, not all projects demand a stable and long-term approach. It seems to me like you were just messing with automating some of the applications you happened to have on your computer, and MSN is of course one of the more interesting ones. Microsoft decided that when they built the MSN client it should not be made easy for anyone to try and automate the application. I think this was mostly done to prevent spam. In the early days it worked, because other people hadn't yet publically documented the protocol. And thus anyone who wanted to spam now had to make a costly investment.
If you do decide to continue with automating the official MSN client, or any other application that resists automation, there are a few tricks that you can try:
ControlSend without specifying the control parameter (use empty string "")
ControlClick on the window with specifying coordinates with using empty string for control parameter again
Eventually, simply Send/MouseClick with optional BlockInput
There are a lot more combinations that you can try (especially ControlX functions) that sometimes miraculously will work, but remember that in the end they are all just window messages. You can do everything in automation with the _SendMessage function, provided you know what you're doing.
Certain designs and interfaces make it very difficult, if not virtually impossible, for AutoIt to tap into the interface to read control information. I would suggest either using the WLM protocol manually via your own client, or utilize a client that doesn't make screen grabbing the text hard or impossible.
Ok, maybe finally an answer... at least this test app I coded works as following:
Somewhere in the beginning of your script you have to set the state of your WLM to #SW_HIDE. Later when you try to do ControlXxxx-stuff you first set a variable on your active window. Now you can optionally block your input and activate your hidden window. It now works as you described above and afterwards you reactivate your last active window. Optionally undo the input block again.
That should solve your question with the least breaktime... you could play with your Mouse Cursor Style in the meantime to simulate some CPU load, so the block input is not too offending. But the action is so fast you won't even recognise it... (maybe it will swallow some keypresses though)
Give it a try!
Opt("WinTitleMatchMode", 2)
#include <GuiConstantsEx.au3>
$GUI = GUICreate("Beispiel HiddenApp", 392, 323)
$ed = GUICtrlCreateEdit("nix", 1, 1, 390, 321)
GUISetState()
Sleep(1000)
GUISetState(#SW_HIDE, $GUI)
WinActivate("SciTE")
Sleep(1000)
BlockInput(1)
$act = WinActive("[ACTIVE]")
WinActivate($GUI)
ControlClick($GUI, "", $ed, "primary", 1, 50, 50)
ControlSend($GUI, "", $ed, "before{Alt}{Tab}after")
WinActivate($act)
BlockInput(0)
Sleep(1000)
GUISetState(#SW_SHOW, $GUI)
While 1
Switch GUIGetMsg()
Case $GUI_EVENT_CLOSE
ExitLoop
Case Else
;;;
EndSwitch
WEnd
Answers.com has a taskbar application that when you ALT + mouse-click on a word in any program it will pop up a window with information pulled from their website.
My question is-- what are the actual programming mechanics and APIs used to do something like this? I don't have Windows application programming experience and am trying to figure out where to start. How do you access the current word pointed to by the mouse?
Anyone aware of any examples or open source software that does anything like this?
It's been a while and the last time I did something like this it was within my own wysiwyg editor so I had full access to all font characteristics needed to calculate which word was clicked by the mouse.
Maybe there's a n easy way to do this if all your apps are .NET or com or share some other framework which provides a way to retrieve this directly.
Via the API, I would look into hooking the keyboard and mouse messages so that your app can pre-process every mouse click on other applications - start with SetWindowsHookEx and read everything you can about hooking messages.
After getting your app to pre-process the messages, you then need to grab the text being clicked. Since text can be painted onto a device context in many different ways, you may be best off doing a screen scrape of the clicked area because the text may only exist as a bitmap. If this is the case, you have to perform some OCR to translate the scraped bitmap back into text. In other cases, the text may reside in the window as text - the WM_GETTEXT message may return this text from some types of windows (e.g. textboxes, buttons, etc.) but for normal windows, this message only return the title in the caption bar.
Sorry I don't have any definite answer, but this may get you started in the right direction.