Sample code for using mac camera in a program? - cocoa

I'd like to use the camera in my Macbook in a program. I'm fairly language agnostic - C, Java, Python etc are all fine. Could anyone suggest the best place to look for documents or "Hello world" type code?

The ImageKit framework in Leopard has an IKPictureTaker class that will let you run the standard picture-taking sheet or panel that you seen in iChat and other applications.
If you don't want to use the standard picture-taker panel/sheet interface, you an use the QTKit Capture functionality to get an image from the iSight.
Both of these will require writing some Cocoa code in Objective-C, but that shouldn't really be an obstacle these days.

If you want to manipulate the camera directly from your code, you must use the QuickTime Capture APIs or the Cocoa QTKit Capture wrapper (much better).
The only caveat is: if you use a QTCaptureDecompressedVideoOutput, remember that the callbacks aren't made on the main thread, but on the QuickTIme-managed capture thread. Use [someObject performSelectorOnMainThread:... withObject:... waitUntilDone:NO] to send messages to an object on the main thread.

There is a utility called isightcapture that runs from the unix command line that takes a picture from the isight camera and saves it.
You can check it out at this web site: http://www.macupdate.com/info.php/id/18598
An example of using this with AppleScript is:
tell application "Terminal"
do script "/Applications/isightcapture myimage.jpg"
end tell

From a related question which specifically asked the solution to be pythonic, you should give a try to motmot's camiface library from Andrew Straw. It also works with firewire cameras, but it works also with the isight, which is what you are looking for.
From the tutorial:
import motmot.cam_iface.cam_iface_ctypes as cam_iface
import numpy as np
mode_num = 0
device_num = 0
num_buffers = 32
cam = cam_iface.Camera(device_num,num_buffers,mode_num)
cam.start_camera()
frame = np.asarray(cam.grab_next_frame_blocking())
print 'grabbed frame with shape %s'%(frame.shape,)
It is used in this sample neuroscience demo

Quartz Composer is also a pleasant way to capture and work with video, when it's applicable. There's a video input patch.
Quartz Composer is a visual programming environment that integrates into a larger Cocoa program if need be.
http://developer.apple.com/graphicsimaging/quartz/quartzcomposer.html

Another solution is OpenCV+python with a script like:
import cv
capture = cv.CaptureFromCAM(0)
img = cv.QueryFrame(capture)
Cannot do any simpler!

Related

How OBS record hidden window?

I'm making a program to record a window that is obscured by another window via python and WIN32API library.
Through many searches, I succeeded in capturing the hidden window through hwnd and BitBlt, but the execution time of my code is not stable.
I tried to provide the recording function by selecting 30~60 fps, but the time required to capture the hidden window and write() it to the video object of cv2 is irregular, so I can't make a 60fps video.
So I thought of OBS and Discord. In the case of OBS, it is possible to enforce stable recording for obscured windows. For Discord, there is a feature that allows you to select a specific window and share it with multiple people in real time (this can also be done for hidden windows).
I'd like to know how these programs provide stable video for occluded windows. I'm a student, and I'm not elite. I am asking this question because it is difficult to analyze the vast Github source code of OBS. Can someone give me an explanation of how the above program captures the screen?
Last time I checked, OBS was doing it with low-level hacks instead of APIs.
Specifically, they have wrote a DLL which they inject into the target application using CreateRemoteThread WinAPI. Then, they patch application’s code to intercept calls to IDXGISwapChain.Present method. Once a call is intercepted, the injected code has access to D3D frame buffer texture. It can copy that texture into another texture on GPU, and then do something with the copy. One possibility is DXGI surface sharing to pass the copy from the target application into the capturing process. The APIs for that don’t require both sides of the sharing to be in the same process, textures can be shared across processes just fine.
Unfortunately for you, their approach is borderline impossible to re-implement in higher-level languages like Python. Such things are only doable in C++ or similar low-level languages, and relatively hard to implement and debug.
#dy.kim, don't be afraid of large codebases. window-capture.c and the OBS GUI fairly obviously list "bitblt" and "Windows Graphics Capture" as the two methods it uses to capture windows, with the preference going to WGC if neither is specified.

3D human Hand design and control via Arduino

I want to design a 3D human hand and control it via signal generated from my Arduino kit. I designed a 3D hand in Blender but how to give the signal generated from Arduino to add life into it. Which tool I should use.
For example I have designed an arbitrary frequency generator. And I want at a particular frequency the Hand will mimic Pinching, or to Fist. Which tool I can use to use the generated signal as input to a Programming interface and output of the Program as a Animated 3D hand.
Please help guys......
Thanks in Advance.
Python would seem the obvious solution to this, as it can interface directly with Blender.
I'm not sure how your controller works with frequencies, but chances are there's a library or a way to handle it in python.
I would suggest looking at this forum post to learn how to set up the animation you want via python script.
Python can then be used to render a series of images (of the hand) like this:
for i in range(last_frame):
bpy.ops.anim.change_frame(frame = i)
bpy.data.scenes['Scene'].render.filepath = '/home/user/Pictures/frame%d.jpg'%i
bpy.ops.render.render()

OSX: automated (every 1-2sec) screenshot (not full screen but (x,y,w,h)) using python

I want to make screenshots on OSX using python. I dont want make full screen shots but only certain rectangles on the screen. Something like (291,305,213,31). I need the correct pixel because afterwards the image files are processed by OCR (python-tesseract) to extract the text.
By the way this is since 6 years the first time I am programming, so far I only know Java a bit. I started yesterday and gave up this morning at 4am. So basically I have no clue yet...For example I still cannot build with Sublime because of path settings, but thats a different story. Cant figure out everything on one day.
I was trying already the following:
- wxPython
But the result are black images, see also:
stackoverflow.com/questions/8644908/take-screenshot-in-python-cross-platform
Additionally it only works in 32-bit mode, but when I do OCR using python-tesseract openCV requires 64-bit....
autopy
when trying to install I got errors, see also:
stackoverflow.com/questions/12993126/errors-while-installing-python-autopy
ImageGrab
only Windows
effbot.org/imagingbook/imagegrab.htm
commandline screencapture
os.system('screencapture test.png')
When I found this I thought, nice but only fullscreen when checking man screencapture. But then I found this: guides.macrumors.c om/screencapture
-R capture screen rect
That would be already enough, but on OSX 10.7.5 I dont have this option. Any ideas?
import Quartz.CoreGraphics
neverfear.org/blog/view/156/OS_X_Screen_capture_from_Python_PyObjC
Create screenshot as CGImage
image = CG.CGWindowListCreateImage(
region,
CG.kCGWindowListOptionOnScreenOnly,
CG.kCGNullWindowID,
CG.kCGWindowImageDefault)
Unfortunately the image is not in file format but a CGImage, no idea how to save as file.
So if possible I would like to use the commandline screencapture with -R if somebody knows how. Just as a start to continue.
Are there any other command line tools available?
What about other libs that I have missed?
Cheers
M
Given that you can get a CGImageRef, you can get its pixel data using the techniques described in Technical Q&A QA1509: Getting the pixel data from a CGImage object. In particular, it shows a function to get the pixel data as a CFDataRef using this function:
CFDataRef CopyImagePixels(CGImageRef inImage) { return CGDataProviderCopyData(CGImageGetDataProvider(inImage)); }
and says:
The pixel data returned by CGDataProviderCopyData has not been color
matched and is in the format that the image is in, as described by the
various CGImageGet functions …
It shows an alternative for getting the pixel data in other formats if you need that.

Trim bar like the GarageBand and iMovie ones

I'm creating a Mac app which needs a trim bar like the GarageBand and iMovie ones. I need a library to get the audio waves to draw them on a NSView. Anyone knows a good library for that?
I think he was referring to the track editor (cut/copy/paste/move).
There's a cool blog post on how to draw Waveforms by the author of the Capo audio editing software:
http://supermegaultragroovy.com/2009/10/06/drawing-waveforms/
I wrote Cocoa (Mac) code based on that, and it's not too hard. You can find that code here:
https://github.com/uliwitness/UKSoundWaveformView
Though it's far from a finished, shippable editor, it's under a permissive license (zlib) and could easily be used as a basis for a full editor.
Maibe this helps: Drawing waveform with AVAssetReader
and if the issue is to draw, then this may help: Parametric acceleration curves in Core Animation
This framework might help you out. From the examples it looks like its pretty easy to use and works well. It provides components to open an audio file, play it and draw the waveform. You might need to implement the cut/copy/paste features on your own.

what language/libraries an app that has a video preview window?

I want to make a simple assistant for putting together AviSynth scripts. This would be a windows desktop application that would have a "preview" screen of an avi movie, which would give you a timeline, play, fast-forward, rewind, advance and go back frame-by-frame. The program would need to know the frame number of the current frame in the player and its filename.
What language is best suited for this? I know PHP ( I understand that this is not a contender ) and am familiar with Java. My thought is that the biggest hurdle with this project will be finding a library for the video playing features. With a cursory glance, no Java video libraries jumped out at me. My next thought would be c++ for this.
The output of this program would be an AviSynth script, a plaintext file which looks like this:
AviSource("myAvi.avi")
Crop(0, 0, 320, 240)
Blur(0.1)
There are a few tool kits that can do tihs:
C#: DirectShow (DirectX)
Java: JMF
If you have Avisynth installed, the only thing you need for preview (If I understood, that's your need) is something that can decode uncompressed video. It would open like a normal file. I'm sure there are video players implemented fairly well in Java, but I don't know how much functionallity from them you need. Anyway parsing scripts is not easy - I recommend you not to try to if you don't need to.
EDIT: I'm sorry, I thought you needed a very specific app, but from what you seem to need, you don't need to code anything, use AVSP!
Please watch this video, it shows how straightforward it is. It has advanced functions such as auto-completion, (even from your own auto-loading scripts!) syntax coloring, macros, automtic importing, drag&drop (of a video, for instance - just drag it to the video and AVSP makes the loading) scrit preview with zoom and all stuff, you can use automatic or custom sliders (you can make a slider that re-writes a number on the script in real time, for instance for hue/luminosity/contrast/etc. that would be cumbersome to control via script), checkboxes & radio buttons (for boolean values, etc...), text fields that alter strings in real time, and basically anything you need... Please check it out.
Also, VirtualDubMod is OLD.
And yep, AVSP is free, both gratis and libre! =)

Resources