I am trying to create a code that takes series of screenshot of a game, use Open-CV to edit it, and display it in a window, all in real-time. I am using windows operating system. For taking screenshots I have tried almost all the methods answered in this question. None of them gives FPS of more than 30, but do affect the performance of the game.
I found out that for games OBS Studio uses a different method called D3d shared texture to capture frames from the game directly on the GPU..
I searched for shared texture in OBS github code but I find the results confusing.
Searching on search engine does not give relevant results.
Can someone explain what is this shared texture method and how do I implement it in my code?
Related
I'm making a program to record a window that is obscured by another window via python and WIN32API library.
Through many searches, I succeeded in capturing the hidden window through hwnd and BitBlt, but the execution time of my code is not stable.
I tried to provide the recording function by selecting 30~60 fps, but the time required to capture the hidden window and write() it to the video object of cv2 is irregular, so I can't make a 60fps video.
So I thought of OBS and Discord. In the case of OBS, it is possible to enforce stable recording for obscured windows. For Discord, there is a feature that allows you to select a specific window and share it with multiple people in real time (this can also be done for hidden windows).
I'd like to know how these programs provide stable video for occluded windows. I'm a student, and I'm not elite. I am asking this question because it is difficult to analyze the vast Github source code of OBS. Can someone give me an explanation of how the above program captures the screen?
Last time I checked, OBS was doing it with low-level hacks instead of APIs.
Specifically, they have wrote a DLL which they inject into the target application using CreateRemoteThread WinAPI. Then, they patch application’s code to intercept calls to IDXGISwapChain.Present method. Once a call is intercepted, the injected code has access to D3D frame buffer texture. It can copy that texture into another texture on GPU, and then do something with the copy. One possibility is DXGI surface sharing to pass the copy from the target application into the capturing process. The APIs for that don’t require both sides of the sharing to be in the same process, textures can be shared across processes just fine.
Unfortunately for you, their approach is borderline impossible to re-implement in higher-level languages like Python. Such things are only doable in C++ or similar low-level languages, and relatively hard to implement and debug.
#dy.kim, don't be afraid of large codebases. window-capture.c and the OBS GUI fairly obviously list "bitblt" and "Windows Graphics Capture" as the two methods it uses to capture windows, with the preference going to WGC if neither is specified.
I tried to sign up, but I was unable; perhaps a problem from my side. Hopefully I'll get an answer as anonymous.
I apologize for the grammar/syntax, but English isn't my native language.
Recently I lost my job, so I have enough spare time to try something fun. I decided to create a simple text RPG game for me and some friends. It will very close to the board games like Talisman, Dungeon Run, and HeroQuest, using dice and a simple attribute/skill system. So no 3d graphics. The only 2d element, if I decide to include it, will be a map
that will allow the hero to move between locations. Currently I'm using Windows XP SP3, for the game I use wxDev-C++, and although cross platform would be cool, I don't really care.
I have some experience in C++ (currently using wxDev-C++), but I'm far from being called an expert or even a great programmer. I was about to start writing parts of the code, but I decided to check if creating a GUI for the game is possible. In some forums, many suggested I use Qt, CEGUI or wxWidgets, but most examples I saw are grey boxes that are
indifferent at best, when I want something that fits better in a fantasy setting. I don't claim I would do better, but I want a GUI that is more fantasy related.
What I want from the GUI:
1. A "cool" Gui with decent graphics. I could even create an image to serve as a mask in Photoshop, but the GUI builder will have to support imported images.
2. A relatively large textbox in the middle (with a scrollbar) that will display die rolls, damage and options.
3. The ability to display dynamically values (like the change in the health after each action without requiring to refresh manually)
4. Display an icon or a small image of the character in the area where I display stats/abilities.
5. Open new windows created with tha same GUI builder to allocate points, buy/sell things and open a map.
About the map in the game: I decided to create a map in photoshop. When the hero decides to move to another location, a new window will open showing the map. I thought of 2 possible ways to move between locations: 1) Create hotspots on the image and select one by clicking on the name of the location.(I dare not think about the complexity of this so we
move to idea #2) and 2) Have the image as a backgroung to a grid with vertical and horizontal coordinates. When the hero selects a new area to visit, he clicks on the area, but what he really does is click on the grid, which returns the two values (x,y) of the location and informs the game about the area the hero wants to visit.
Yeah, yeah, I know it's too much, so what I'm most interested in are the 1-3. I know that even if they are possible, it will propably take forever, but as I said I have spare time, and I like learning new things. I apologize for the size of the post, but I decided to post as many info as possible so you know what I want.
If any of you has used Qt, CEGUI or wxWidgets could you tell which covers most of my criteria? I saw some great stuff build with CEGUI, but I don't know if it is too hard to learn?
Thank in advance.
I know my answer comes pretty late, I only recently started using stackoverflow fairly recently, but maybe this response will help anybody.
CEGUI fully supports skinning widgets using XML. Our CEED editor (WYSIWYG) fully supports layout editing, but the skinning editor (LNF editor) is not finished as of now (11.11.2014), the development version supports exchanging images however and changing sizes and proportions, but more advanced adjustments have to be done in XML.
CEGUI has an imageset editor, fully supported by the CEED editor. Creating imagesets (sets of named subimages, with position and dimension inside a big texture atlas) is supported there. Additionally there is a way to create imagesets from just a bunch of jpg/png/... files using a tool. You would have to ask for specifics in the forum though because it is not integrated into CEED yet.
So basically with CEGUI you are free to make whatever fantasy GUI you want. Skinning simple elements like buttons and progress bars isn't much work in XML anyways. Without the finished editor, some more advanced widgets are more work to skin, but many skins have already been created done this way and some of them are even publically available in the forum and in the CEGUI stock files.
StaticText widgets supports what you want, you can even use images in there or change fonts and colours in the text if you want. Scrollbars are supported too.
I am not sure what you mean by this. You have to specify this.
A simple "Generic/Image" widget is available in CEGUI for this purpose. You can use precreated images or even RTT textures.
You can create and destroy windows in CEGUI without issues.
Regarding the map: I m not sure what you mean, but getting the position of a click in respect to an image (representing the map) is possible in CEGUI.
CEGUI is not particularly hard to learn. There is always the forums and the chat if you got questions. For an Open Source project it is quite well documented so if you read all of the API docu, and look at the supplied samples in the sample browser, you should already get quite far. And for everything additional there is the forum (search), the IRC chat and a community wiki (mind the targeted versions of an article there though)
For a project like yours, CEGUI seems perfectly suited (this is what it was created for in the first place). Qt is not really optimal for games for numerous reasons. wxWidgets I have never used.
I have a scenario that I need some good solid advice on. The question is really about speed of WriteableBitmap vs. images in IsolatedStorage on the Windows Phone.
I have an app that displays a UserControl (#1) which is a little graphically heavy. When the user swipes it, it transitions in a push-left type of transition to bring in a new UserControl (#2) which is also a little graphically heavy. If the user swipes the other way, control #1 is brought in in the same type of push-transition, this time from the right.
What I do today is take a snapshot of #1, load #2 off screen and take a snapshot of it, put both side-by-side in a Canvas control and animate that control either left or right. One of the reasons I don't just use the controls and animate them is they may have animation that starts when they are loaded - my current technique allows me to capture a screen shot of pre-animation and post-animation, depending on which direction they go in.
What I'm wondering, however, if it would be better/faster to just do the above the first time and send the writeablebitmap to IsolatedStorage with Extenstions.SaveJPEG and just use that instead in subsequent tranistion animations.
Would load/render/WriteableBitmap each time generally be faster or load jpeg from IsolatedStorage be faster each time? I see that the Transitions control in the SDK doesn't really do either of these, so I'm open to suggestions that are different that also might improve performance.
I expect this to be very depended on the hardware and application. So it is pretty hard to give an answer based on this input. It doesn't look to hard to test (on actual hardware and with the actual application) so my advice is to build both and test.
The applications I have been working with use both approaches and to be honest I haven't noticed much difference.
Also you might try and enable bitmap caching on the controls. This will give you a writeable bitmap implementation that is very fast.
I am writing a DLL that needs to do some work in Cuda 3.2 and some work in OpenGL. OpenGL will render some grayscale images that my Cuda code needs to read in and modify, and then give back to OpenGL as a texture. I believe I need to create PBOs to do that. I have done some basic OpenGL stuff before but never worked with extensions, and that's where my problem is - I've been searching for 2 days and so far haven't been able to find a working example, despite wading through pages and pages of code. None of the samples I've tried work (and I'm sure my vid card will support it, being a GTX470)
Some specific questions:
1. I installed the nvidia opengl sdk. Should I be using glew.h and wglew.h to access the extensions?
2. My DLL does not have any UI - do I need to create a hidden window or is there an easier way to create an off-screen rendering context?
3. Can I create a grayscale PBO by using GL_RED_8UI format? Will both cuda and gl be happy with that? I read the opengl interop section in the cuda programming manual and it said GL_RGBA_8UI was only usable by pixel shaders because it was an OpenGL 3.0 feature, but I didn't know if that applied to a 1-channel format. 1 channel float would also work for my purposes.
4. I thought this would be fairly easy to do - does it really require hundreds of lines of code?
Edit:
I have code to create an OpenGL context attached to a HBITMAP. Should I create a bitmap-rendering context and then try to attach a PBO to that? Or will that slow me down by also rendering to CPU memory? Is it better to create an invisible window and attach the PBO to that? Also, does the pixel format of my PBO have to match the window/bitmap? What about the dimensions?
Thanks,
Alex
There's actually an example of how to use OpenGL and CUDA together. Look at the SimpleGL example.
You may want to take a look at this example:
https://github.com/nvpro-samples/gl_cuda_interop_pingpong_st
I'm developing my first Android game and I'm having a bit of difficulty making the UI as smooth as I would like. I've spent a couple of hours googling around with no luck, I'm probably just searching for the wrong thing.
I have two different XML layout resources where each layout contains just one SurfaceView subclass. When I call activity.setContentView(R.layout.second_layout) to transition from the first layout to the second layout there is a noticeable period of time where a black screen (with a small white bar along the top) is displayed in between the two views.
I've tried various things such as; constructing the second view manually at runtime (i.e not using a layout XML file), calling activity.overridePendingTransition(android.R.anim.fade_in, android.R.anim.fade_out) after activity.setContentView(R.layout.second_layout) and attempting to render to the canvas before the view has loaded (turns out the canvas is unavailable).
I don't see other games (or apps) having this issue so I presume there is a reasonably simple solution.
If you need some more information about my particular situation in order to help out then please let me know what information is missing. Any help would be largely appreciated.
Update: My answer below was written in 2010. Since then Fragments have become the norm, particularly since Fragment nesting was made possible and the support library allows this functionality to be used in a backwards compatible fashion. As such, instead of transitioning to a new Activity to perform a new "user task", you can use the one Activity and push and pop fragments within that Activity's view hierarchy. Animations can also be performed as a part of a fragment transaction (e.g. Fragment transaction animation: slide in and slide out).
This became pretty apparent not long after posting this question, however I thought I should come back here and make it clear to everyone else.
Activities are positively the way to go when developing for Android. Don't be put off by the fact that a transition may seem too minor for a separate Activity, the very foundation of Android is built around the idea of an Activity.