I want to control the mouse pointer with my application and be able to interact with other programs using my program,
For example I want my application to be able to click on a button on another application
How should I go about solving this problem?
(Any programming language would work, also if you have any suggestion please let me know)
Afterthoughts:
I want to do it in windows operating system and want to test my GUI to see if it works in different scenarios. Any language would work for me since this is not part of the final product but I prefer one of these languages (Python, Java, C# or MATLAB)
Thanks
There are many ways of doing this, and you didn't mention any details of your application (system, target goal, etc...).
If your goal is menial automation, I'd recommend whipping together a quick AutoIt script on Windows. http://www.autoitscript.com/autoit3/index.shtml
If this isn't what you're looking for, please give more details.
Okay, this one is really operating system and windowing specific. But the phrase you're looking for is "mouse grabbing".
As #Mitch suggests, unless you've got a really good reason — like maybe a GUI testing app? — then grabbing the mouse and messing with it in that way is very bad form.
Related
I want to clear the console
I tried using this, but it doesn't work for Windows
print!("\x1B[2J");
Is there an easy way to clear the console?
I think I need some clarification first, on what you're attempting to do. Regardless, this is my best attempt at answering your question <3
Preamble
Given that different terminals work differently and have different APIs, I think you probably will want to use a crate that provides this functionality in a cross-plaform manner.
I'm assuming from here on that you want cross-platform functionality and that you don't mind using external dependencies. If this is correct, you might be happy to know that the terminal interaction crates are actually really developed in the Rust ecosystem. I have heard only good things about them.
With that out of the way, let's move on.
Do you want to just perform actions on the terminal?
If you just want to perform some actions on the terminal, like "clearing", "scrolling", "moving the cursor" and whatnot, I think you will be satisfied with the terminal crate. It allows to perform many actions, like clearing, independently of the platform you're in. It also allows for using interactivity features like interacting with the Mouse and the Keyboard :3
Or do you want to write a GUI for the terminal?
If what you want to do is write a Console-based User Interface though, I think that what might work for you instead is the tui crate. It has all of what you need to build terminal GUIs, from clearing of the console up to graphical widgets. Tools like gitui are written with tui.
Did I answer your question? Feel free to follow up if I fell short :)
I've found a way to clear the console in rust
By using console crate, console.clear() method
i've been challenged to write an app simulating keystrokes. After pressing shortcut app should be able to send predefined key combination to currently active app. It is functionality provided by many existing applications, but i want to write it on my own. App should be usable on windows.
Could you provide me with suggestions about:
which programming language should I choose?
Is there any libraries providing such functionality?
EDIT: To be precise: both applications are standalone, windows app.
The native winapi for doing this is SendInput.
To answer your questions:
Which programming language? This is up to you. Because this is just a simple native API call, use any language that allows you to call native API.
Don't bother with libraries if this is all you need, because it's very simple.
Now, to go further, I know you didn't ask this but many many people continue by asking how to send keystrokes to windows that don't have keyboard focus - like sending keystrokes to a specific app. This is much more difficult and error-prone. And since it goes outside the behavior of actual real keystrokes, it can behave unpredictably. Here is one such question.
You can easily do this on C#, using
SendKeys.Send("key here");
or
SendKeys.SendWait("key here");
Some keys use other keycodes, you can see them here on MSDN.
There are many applications for Windows these days that don't use native windows controls, don't have standard window frames and generally look different. What are some recommended techniques for creating such interfaces?
There are good reasons not to. Like that you will most likely not do a better job than Windows does. (Maybe it will look better (in your opinion), but will it behave?). Or that it's not what most users expect. Or that it will look like s**** on Windows 2011.
That said, it's not hard. You simply handle the WM_NC* events like WM_NCPAINT or WM_NCHITTEST. NC stands for Non Client (window area). And of course, there is a trick on Vista/Win7 (you have to announce it to the DWM).
From an implementation aspect, you could employ WPF (Windows Presentation Foundation) assuming you code for .NET :) It has pretty bunch of skinnable controls, that may look like native and may not.
From a design aspect, if your interface isn't going to follow documented standards (like the Windows UI guidelines), it has to be intuitive. I think the new generation of Windows applications will go through a growing phase in a manner similar to the early days of the Web. After a time, some standards or common themes will evolve.
Can you give us some sample applications? Some apps that don't use native windows controls use cross-platform GUI libraries, like Qt for C++ or Tkinker. These maintain the same look across different platforms.
I wouldn't really recommend making your user interface different deliberately. You don't stand to gain much. Your controls are almost always going to be buggier than native controls, and you are requiring the user to learn something new. Now, if you're controls add a large enough value to be worth the users' time it can be okay. But making them get used to different looking buttons is rarely worth it.
I`m not sure if this answer your question.
You can use third party skinning controls like from Infragistics, or SkinSoft for example.
But like Bubba said I`d recommend going for WPF.
Model-View-Controller! It's as valuable here as in web apps or anywhere else. Be sure to keep the part of your program that generates the custom UI separate from the part of your program that flashes the BIOS.
I know this question is 10 years old but none of the answers mention using an option in visual studio, dont know if it existed at the time.
Theres an option to remove the border of the window in visual studio (called borderStyle). Thats the easiest way to do it, using C#. After removing the border, all you have to do is create a new interface. If you're looking to do it in C++, i think you need to use DWM. I will let an example i found here.
https://github.com/melak47/BorderlessWindow
Another example (maybe without DWM? didnt test):
https://social.msdn.microsoft.com/Forums/vstudio/en-US/b98c4c06-9581-44d3-8e5a-4adb2316e653/win32-about-styles-how-can-i-do-a-borderless-window?forum=vclanguage
There is a lot of people disencouraging to do it in this thread but there's no reason to not do it, if you know what you're doing your application can look great.
So I have this program that I really like, and it doesn't support Applescript. I'd like to automate it a little bit. Now, I know that I could use applescript to tell the program to tell the menu to tell the submenu to tell the menuitem to activate or whatever, but frankly I don't like applescript very much anyway.
When I open the NIB file in IB, I can see the messages that are being sent to FirstResponder; for example, the Copy menu item sends "copy:". Is there any way for me to invoke this directly from another program?
No. It's called protected memory for a reason, you know. The other program is completely insulated from your application. There are ways to put code into other apps, but (a) it's very inadvisable (b) requires root privileges, which means the rest of your app needs to be ROCK SOLID AND IMPREGNABLE, and (c) writing such code is a black art requiring knowledge of the operating system kernel interfaces, virtual memory management, the ABI, the internals of the linker/loader, assembler programming, and the operational parameters and other specifics of the particular processor upon which your app happens to be running.
Really, AppleEvents and other such IPC mechanisms are there for a reason.
Your other alternatives (all of which are a bit hacky, to be honest, and give you the fairly significant burden of ensuring the target app is in the state you want/expect) the access the data you're looking for are:
The Accessibility APIs from the ApplicationServices framework, through which you can traverse the UI tree to grab the text from wherever you need it directly, or can activate the menu item. Access for your app has to be explicitly granted by the user, however (although this is much the same as the requirement for UI scripting).
You can use the CoreGraphics APIs (within the ApplicationServices framework again) to send keyboard events to the target application (or just to the system) directly. This would mean sending four events: Command-down, C-down, C-up, Command-up.
None of these are ideal. To be honest, your best approach would be to look at your requirements and figure out how you can best engineer around the problem by changing those requirements in some way, i.e. instead of grabbing something directly, ask the user to provide some input, etc.
You might be interested in SIMBL or in mach_inject. SIMBL is a daemon (in my fork based on mach_inject, in the original version based on injection via some ScriptingAdditions hack) which does the injection for you, so you just need to put a bundle with your code into the SIMBL directory and SIMBL will inject it for you into the target application. Or you can do so yourself via mach_inject. Or probably more convenient, mach_inject_framework which injects and runs code which just loads some framework.
I think Jim may overstate the point a bit; he's not wrong, but it seems misleading. There are lots of ways to cause a Cocoa program to execute its own code under you control (Carbon is harder). The Accessibility API is very commonly used this way (so commonly that I expect it to be repurposed eventually). Fscript can give you all kinds of access to the innards of another Cocoa program. While Input Managers may well exit the scene at some point, SIMBL is still out there today to do this kind of stuff.
Whether you like Applescript or not, Apple Events are the primary way Apple provides for inter-program control. Have you double-checked Script Editor's Open Library function to find out if the program really does have any Applescript support? You can code Apple Events entirely in Objective-C these days using Leopard's Scripting Bridge. I wrote up a tutorial if you like (it's still under-documented by Apple).
Cocoa is a reverse-engineer's dream. The same guys who host SIMBL have a nice intro to the subject. "Wolf" also writes a lot of useful information on this.
Jim's right. Many of these approaches can completely destabilize the system if done incorrectly (sometimes even if done correctly). I don't do much of this stuff on my production systems; I need them to work. But there are a lot of things you can make a Mac app do, and it's a good part of a Mac developer's training to understand how all the pieces really work.
We wish to provide people to be able to add some logic to their accounts(say, given a few arguments, how to compute a particular result). So, essentially, this would be tantamount to writing simple business rules with support for conditionals and expressions. However, the challenge is to provide them a simple online editor where they can create the logic (preferably) by completely visual means (drag/drop Expr-tree nodes maybe -- kinda like Y! pipes).
Does anybody know of a scripting/macro/domain-specific language that lets people do this? The challenge is the visual editor, since we don't wish to invest in developing the UI to do the editing. The basic requirements would be:
1. Embedded into another language, or run securely (no reboot -n or <JUNK-DANGEROUS-COMMAND> >> ~/.bashrc)
2. Easily accessible to users without coding background (no need of any advanced features)
3. Preferably have a simple GUI based editor to create the logic programs accessible to non-developers (kinda like spreadsheets)
4. Some ability to generate compile-time warnings (invalid code) would be good (Type safety?)
5. Ability to embed some data before execution which is available to the interpreter (Eg., name, birthday, amount)
Anybody tried doing something like this and got any ideas? I looked at Lua, Io, Python, Ruby and a host of others, but the challenge essentially is that I don't think non-programmers will be able to understand the code all that much. Something that could be added via "meta-programming" to say a Ruby would be good as well, if an editor could be easily developed!
As a matter fact, Microsoft is developing Oslo, which is right up your alley.
Chris Sells has been writing a lot about it recently.
It is designed to be a way to author DSLs and also to visually author these models with a graphical tool called Quadrant. Sounds very very similar to what you are looking for.
Open source wise, Ruby I think can be close, as you can see if you look at _whytheluckystiff's Try Ruby or Hackety.
I don't think you'll find anything that isn't too generic, especially regarding the GUI editor. There's no generic tools as far as I know that will be able to automatically interface with your program and be able to query data from it and interpret the script into commands in your software -- if there is I'd like to have a copy. Not being flippant, but you will have to do some (probably alot) of work to get this working. It will probably result in you writing a custom DSL.
I would take a look at PowerShell. You could surface all the activities a user would like to script in a very readable way.
There is some talk of using PowerShell to create a DSL on the PowerShell team blog and Bruce Payette, the technical lead, talks about this in his book Windows PowerShell in Action from Manning.
At the other end of the scale is to write something simple as a HyperText Application (HTA) -- assuming Windows of course -- along the lines of my Clive tool. The article on the blog doesn't mention the HTA version, but essentially I could enter VBScript-ish code into one textarea and interpret it on the spot, output going into another text area on the form.
With HTAs giving you all the form control of HTML, plus the DOM, you could come up with something interesting fairly quickly.