Before I get shot down on this one, I realize that the 'how' answer for this question might be slightly debatable, however I'm more interested in the 'what'.
In a nut shell I want to know which methods I can use to interact with a PC video game interface. I want to create a program that can extract data from a video game market interface.
My first initial thought was that I would need to programmatically take screen shots and then use some Optical Character Recognition software to extract the text. Then run whatever operation on the extracted text to derive my incites.
Then I was thinking it might just be easier to have a bunch of mini screen shots that I just use to find matches on certain sections of the screen. When a match is found, I would then know what the text is on the screen, without having to actually 'extract' it.
For those out there whom have done this, can you point me in one direction or the other? Perhaps there is a method that I am completely unaware of.
If its the case that this question is not suitable for this forum. It would be much appreciated if you could direct me elsewhere.
Edit: I should probably add that I'm not looking to spend a fortune on this project... so any free software would be the best. Perhaps that's a tall order.
I'm starting to think Sikuli is the direction I'm going to go. Open Source image recognition software, integrates with Python, Ruby, Java, JDBC, JavaScript and more.
-- Expanding on the question --
There are basically 3 categories of tools:
Recorder while you manually work along your workflow, a recorder tracks your mouse and keyboard actions. After stopping the recording, you might playback (autorun your worflow). The recordings can usually be edited and augmented with additional features.
GUI aware the tool allows to programmatically operate on GUI elements like buttons. This is based on the knowledge of internal structures and names of the GUI elements and their features. Some of these tools also have a recording feature.
Visually the tool “sees” images (usually retangular pixel areas) on the screen and allows to act on these images using mouse and keyboard simulation. There might be some recorder feture as well with such a tool.
SikuliX belongs to the 3rd category and currently does not have a recorder feature.
Answer in progress...
In games with moddable UIs, like many MMOs, you could create a mod that streams data through a series of black and white squares that could be read with optical sensors. From there, a microcontroller could deliver the data back to the PC via USB or wifi.
My approach as a noob. First determine if OCR 100% needed, I think this plays a role in speed.
if possible:
-run game in window (allows for trouble shooting and easy troubleshooting)
-is there a high contrast option for game? Will help Sikuli find things
then you plan out your scenarios:
You have to create different functions for different situations. A lot of gaming is "do you see this?" Then "do this" until that is gone.
Start with small parts you want to automate then build on them. Making sure your parts can scale in case small change need to happen, they will. For instance you want to open the menu if you see an object, lets say a tree.
Assume you have some sort of walking algorithm.
setROI(region1) #focus here for tree
if exists(tRee):
click(loCation) #you could hit the shortcut key to opening the menu
click(iTem) #if the item moves in the menu then you may need to scroll to find it first or you can change the ROI and start seeing if sikuli can differentiate your item from one you dont want to click.
You would get that to loop into other actions and proceed. Goodluck.
Related
I am new to coding and wanted to get some hands on practice with a project I have in mind. Here it is:
Let's say you have blank page and on the side of a screen you have several items you can choose to draw on the blank page. For example the background can be mountains, the ocean, a forest etc. On top of that you can place a house, a church or another selectable element. Whatever you like.
It is like a picture editor where you can put together a picture with different pre-given elements. Or like in video games where you can create your own character.
What would I need to build a web application for that kind of thing?
This link should get you started but it won't be the complete answer to your question - http://www.webdesignerdepot.com/2013/08/how-to-use-html5s-drag-and-drop/
Essentially, you can achieve your image dragging and dropping using similar techniques. It will require a bit of Spike work from yourself, and looking into how HTML5 can handle drag and drop. I discovered this resource fairly quickly and I think the solution you want isn't as complicated as you may think, it just requires a bit of know-how regarding drag and drop operations within HTML5 :-)
Also, there may already be some JavaScript based API's that do this sort of thing easier but I'm not too aware - I suppose starting this way could be a great introduction for you and you may wish to expand once you've done some work for it :-)
Hope this helps you and your coding journey!
I'm working on a small image processing project in MATLAB. I have worked with MATLAB before, but never created a GUI. The GUI I want to create could be pretty advanced, so I need some hints on how to get started.
The purpose of the GUI would be to load an image and have it shown to the user. The user then has to click on two points in the image, of which the coordinates are stored (in pixels) in a variable. If possible, a colored dot is shown where the user has clicked. After the user finished with the current image, he can load a next one.
I have some experience with Java, and I think this wouldn't be too hard in Swing. But MATLAB seems like not having the purpose of creating such an advanced GUI. However, the whole project until now is in MATLAB, so it would be nice if I could manage to do it. Any help? Hints? Things I should look at?
Thanks a lot.
This is not a very complex task to be done in MATLAB.
For simple instructions about adding a picture to a GUI, take a look at this post:
http://blogs.mathworks.com/pick/2007/10/16/matlab-basics-setting-a-background-image-for-a-gui/
For instructions on various interactions between GUI axes and the mouse pointer, check this video (keep in mind that your picture in the GUI lies within normal MATLAB axes):
http://blogs.mathworks.com/pick/2008/05/27/advanced-matlab-capture-mouse-movement/
In general, Doug's tutorial videos are great for MATLAB beginners, and I'd advise you to take a look at more of them.
I tried to sign up, but I was unable; perhaps a problem from my side. Hopefully I'll get an answer as anonymous.
I apologize for the grammar/syntax, but English isn't my native language.
Recently I lost my job, so I have enough spare time to try something fun. I decided to create a simple text RPG game for me and some friends. It will very close to the board games like Talisman, Dungeon Run, and HeroQuest, using dice and a simple attribute/skill system. So no 3d graphics. The only 2d element, if I decide to include it, will be a map
that will allow the hero to move between locations. Currently I'm using Windows XP SP3, for the game I use wxDev-C++, and although cross platform would be cool, I don't really care.
I have some experience in C++ (currently using wxDev-C++), but I'm far from being called an expert or even a great programmer. I was about to start writing parts of the code, but I decided to check if creating a GUI for the game is possible. In some forums, many suggested I use Qt, CEGUI or wxWidgets, but most examples I saw are grey boxes that are
indifferent at best, when I want something that fits better in a fantasy setting. I don't claim I would do better, but I want a GUI that is more fantasy related.
What I want from the GUI:
1. A "cool" Gui with decent graphics. I could even create an image to serve as a mask in Photoshop, but the GUI builder will have to support imported images.
2. A relatively large textbox in the middle (with a scrollbar) that will display die rolls, damage and options.
3. The ability to display dynamically values (like the change in the health after each action without requiring to refresh manually)
4. Display an icon or a small image of the character in the area where I display stats/abilities.
5. Open new windows created with tha same GUI builder to allocate points, buy/sell things and open a map.
About the map in the game: I decided to create a map in photoshop. When the hero decides to move to another location, a new window will open showing the map. I thought of 2 possible ways to move between locations: 1) Create hotspots on the image and select one by clicking on the name of the location.(I dare not think about the complexity of this so we
move to idea #2) and 2) Have the image as a backgroung to a grid with vertical and horizontal coordinates. When the hero selects a new area to visit, he clicks on the area, but what he really does is click on the grid, which returns the two values (x,y) of the location and informs the game about the area the hero wants to visit.
Yeah, yeah, I know it's too much, so what I'm most interested in are the 1-3. I know that even if they are possible, it will propably take forever, but as I said I have spare time, and I like learning new things. I apologize for the size of the post, but I decided to post as many info as possible so you know what I want.
If any of you has used Qt, CEGUI or wxWidgets could you tell which covers most of my criteria? I saw some great stuff build with CEGUI, but I don't know if it is too hard to learn?
Thank in advance.
I know my answer comes pretty late, I only recently started using stackoverflow fairly recently, but maybe this response will help anybody.
CEGUI fully supports skinning widgets using XML. Our CEED editor (WYSIWYG) fully supports layout editing, but the skinning editor (LNF editor) is not finished as of now (11.11.2014), the development version supports exchanging images however and changing sizes and proportions, but more advanced adjustments have to be done in XML.
CEGUI has an imageset editor, fully supported by the CEED editor. Creating imagesets (sets of named subimages, with position and dimension inside a big texture atlas) is supported there. Additionally there is a way to create imagesets from just a bunch of jpg/png/... files using a tool. You would have to ask for specifics in the forum though because it is not integrated into CEED yet.
So basically with CEGUI you are free to make whatever fantasy GUI you want. Skinning simple elements like buttons and progress bars isn't much work in XML anyways. Without the finished editor, some more advanced widgets are more work to skin, but many skins have already been created done this way and some of them are even publically available in the forum and in the CEGUI stock files.
StaticText widgets supports what you want, you can even use images in there or change fonts and colours in the text if you want. Scrollbars are supported too.
I am not sure what you mean by this. You have to specify this.
A simple "Generic/Image" widget is available in CEGUI for this purpose. You can use precreated images or even RTT textures.
You can create and destroy windows in CEGUI without issues.
Regarding the map: I m not sure what you mean, but getting the position of a click in respect to an image (representing the map) is possible in CEGUI.
CEGUI is not particularly hard to learn. There is always the forums and the chat if you got questions. For an Open Source project it is quite well documented so if you read all of the API docu, and look at the supplied samples in the sample browser, you should already get quite far. And for everything additional there is the forum (search), the IRC chat and a community wiki (mind the targeted versions of an article there though)
For a project like yours, CEGUI seems perfectly suited (this is what it was created for in the first place). Qt is not really optimal for games for numerous reasons. wxWidgets I have never used.
I want to make a simple assistant for putting together AviSynth scripts. This would be a windows desktop application that would have a "preview" screen of an avi movie, which would give you a timeline, play, fast-forward, rewind, advance and go back frame-by-frame. The program would need to know the frame number of the current frame in the player and its filename.
What language is best suited for this? I know PHP ( I understand that this is not a contender ) and am familiar with Java. My thought is that the biggest hurdle with this project will be finding a library for the video playing features. With a cursory glance, no Java video libraries jumped out at me. My next thought would be c++ for this.
The output of this program would be an AviSynth script, a plaintext file which looks like this:
AviSource("myAvi.avi")
Crop(0, 0, 320, 240)
Blur(0.1)
There are a few tool kits that can do tihs:
C#: DirectShow (DirectX)
Java: JMF
If you have Avisynth installed, the only thing you need for preview (If I understood, that's your need) is something that can decode uncompressed video. It would open like a normal file. I'm sure there are video players implemented fairly well in Java, but I don't know how much functionallity from them you need. Anyway parsing scripts is not easy - I recommend you not to try to if you don't need to.
EDIT: I'm sorry, I thought you needed a very specific app, but from what you seem to need, you don't need to code anything, use AVSP!
Please watch this video, it shows how straightforward it is. It has advanced functions such as auto-completion, (even from your own auto-loading scripts!) syntax coloring, macros, automtic importing, drag&drop (of a video, for instance - just drag it to the video and AVSP makes the loading) scrit preview with zoom and all stuff, you can use automatic or custom sliders (you can make a slider that re-writes a number on the script in real time, for instance for hue/luminosity/contrast/etc. that would be cumbersome to control via script), checkboxes & radio buttons (for boolean values, etc...), text fields that alter strings in real time, and basically anything you need... Please check it out.
Also, VirtualDubMod is OLD.
And yep, AVSP is free, both gratis and libre! =)
Many animation effects are simply gratuitous eye candy -- however, there are situations where animations effectively communicate to the user what's going on.
What are some of your favorite uses for animations, and what specific animation type would you use?
E.g.: Animate items downwards when a new item is inserted into a list
I really like Google Chrome's use when a file is being downloaded. It's hard to describe, but, it's a circle that fills like a pie chart as the download progresses, and the circle is overlaid with the icon for the file you're downloading. Very slick.
One example I can think of is the animation used by operating systems when you minimize a window.
Both Microsoft Windows and Apple OS X animate the window going down to the taskbar (or the Dock in OS X) to show the user where the window went. Otherwise novice users that hit minimize by accident might have trouble getting the window back.
I don't use linux, but I'm pretty sure it does the same. I'm not being discriminative =)
From enjoy3d.com
enjoy3d.com http://worldsware.com/images/mouse.gif
Press your mouse button
and move to look around.
There is a very nice paper by Ben Bederson and Angela Boltman in which they evaluate the impact of animation on user’s ability to build a mental map of the information in the space:
Does Animation Help Users Build Mental Maps
of Spatial Information?
I believe that all visual changes should not be swift. Be it status notification, window maximized/minimized, or data deleted/added. I cannot find a reference, but usually it is recommended that all animations should not be around 1-2 seconds, matching human's response time.
My favorite uses of animation is not in a commercial software (though Apple is good at this) but a research paper called Phosphor which I consider one of the great UI ideas that have not yet implemented into major operating systems.
AJAX loading gifs - you've got to have an indicator that you definitely registered an event and you're doing something about it
Progress bars are nice for things that take more than a moment or two, but only when they are accurate. An inaccurate progress bar is worse than none, in my opinion.