I'm building a simple video editor for macOS: A movie file is loaded as an AVAsset, transformed by a series of CIFilters in a AVVideoComposition, and played by an AVPlayer. I present UI controls for some of the parameters of the CIFilters.
When video is playing everything is working great, I slide sliders and effects change! But when the video is paused the AVPlayerView doesn't redraw after the controls in the UI are changed.
How can I encourage the AVPlayerView to redraw the contents of the videoComposition of it's current item when it's paused?
class ViewController: NSViewController {
#objc #IBAction func openDocument(_ file: Any) { ... }
#IBOutlet weak var moviePlayerView: AVPlayerView!
var ciContext:CIContext? = CIContext(mtlDevice: MTLCreateSystemDefaultDevice()!)
var sliderValue: Double = 0.0
#IBAction func sliderMoved(_ sender: NSSlider) {
self.sliderValue = sender.doubleValue
// need to update the view here when paused
self.moviePlayerView.setNeedsDisplay(self.moviePlayerView.bounds)
// setNeedsDisplay has no effect.
}
func loadMovie(file: URL) {
let avMovie = AVMovie(url: file)
let avPlayerItem = AVPlayerItem(asset: avMovie)
avPlayerItem.videoComposition = AVVideoComposition(asset: avMovie) { request in
let output = request.sourceImage.applyingGaussianBlur(sigma: self.sliderValue)
request.finish(with: output, context: self.ciContext)
}
self.moviePlayerView.player = AVPlayer(playerItem: avPlayerItem)
}
}
It turns out re-setting the AVPlayerItem's videoComposition property will trigger the item to redraw. This works even if you set the property to it's current value: item.videoComposition = item.videoComposition. The property setter appears to have undocumented side effects.
To fix the sample code above, do this:
#IBAction func sliderMoved(_ sender: NSSlider) {
self.sliderValue = sender.doubleValue
// update the view when paused
if self.moviePlayerView.player?.rate == 0.0 {
self.moviePlayerView.player?.currentItem?.videoComposition = self.moviePlayerView.player?.currentItem?.videoComposition
}
}
Hopefully someone finds this useful!
Related
I have tried to make an app scanning the QR code within ios 10 and Swift 3. However, my QRScannerController could not detect the QR Code but showing the camera view.
I don't understand what's wrong with the code. Here is the implement of the controller:
import UIKit
import AVFoundation
class QRScannerController: UIViewController, AVCaptureMetadataOutputObjectsDelegate {
#IBOutlet var messageLabel:UILabel!
#IBOutlet var topbar: UIView!
//TESTING
var captureSession: AVCaptureSession?
var videoPreviewLayer: AVCaptureVideoPreviewLayer?
var qrCodeFrameView: UIView?
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view.
// Get an instance of the AVCaptureDevice class to initialize a device object and provide the video as the media type parameter
let captureDevice = AVCaptureDevice.defaultDevice(withMediaType: AVMediaTypeVideo)
do {
view.bringSubview(toFront: messageLabel)
view.bringSubview(toFront: topbar)
// Get an instance of the AVCaptureDeviceInput class using the previous deivce object
let input = try AVCaptureDeviceInput(device: captureDevice)
// Initialize the captureSession object
captureSession = AVCaptureSession()
// Set the input devcie on the capture session
captureSession?.addInput(input)
// Initialize a AVCaptureMetadataOutput object and set it as the input device
let captureMetadataOutput = AVCaptureMetadataOutput()
captureSession?.addOutput(captureMetadataOutput)
// Set delegate and use the default dispatch queue to execute the call back
captureMetadataOutput.setMetadataObjectsDelegate(self, queue: DispatchQueue.main)
captureMetadataOutput.metadataObjectTypes = [AVMetadataObjectTypeQRCode]
//Initialise the video preview layer and add it as a sublayer to the viewPreview view's layer
videoPreviewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
videoPreviewLayer?.videoGravity = AVLayerVideoGravityResizeAspectFill
videoPreviewLayer?.frame = view.layer.bounds
view.layer.addSublayer(videoPreviewLayer!)
//start video capture
captureSession?.startRunning()
//Initialize QR Code Frame to highlight the QR code
qrCodeFrameView = UIView()
if let qrCodeFrameView = qrCodeFrameView {
qrCodeFrameView.layer.borderColor = UIColor.green.cgColor
qrCodeFrameView.layer.borderWidth = 2
view.addSubview(qrCodeFrameView)
view.bringSubview(toFront: qrCodeFrameView)
}
func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputMetadataObjects metadataObjects: [Any]!, from connection: AVCaptureConnection!) {
// Check if the metadataObjects array is not nil and it contains at least one object.
if metadataObjects == nil || metadataObjects.count == 0 {
qrCodeFrameView?.frame = CGRect.zero
messageLabel.text = "No QR code is detected"
return
}
// Get the metadata object.
let metadataObj = metadataObjects[0] as! AVMetadataMachineReadableCodeObject
if metadataObj.type == AVMetadataObjectTypeQRCode {
// If the found metadata is equal to the QR code metadata then update the status label's text and set the bounds
let barCodeObject = videoPreviewLayer?.transformedMetadataObject(for: metadataObj)
qrCodeFrameView?.frame = barCodeObject!.bounds
if metadataObj.stringValue != nil {
messageLabel.text = metadataObj.stringValue
}
}
}
} catch {
//If any error occurs, simply print it out
print(error)
return
}
}
override func didReceiveMemoryWarning() {
super.didReceiveMemoryWarning()
// Dispose of any resources that can be recreated.
}
Your message label is going under your view controller view. Just bring the message label to front at the end of view did load would help.
I have created a sample project and it was working fine on iPhone. Please take a look at here
First of all I would like to thank anyone in advance for any help I get.
I have searched far and wide across the net and cannot find a solution to my issue. My issue is with an iOS game I am building using the SpriteKit framework. I have added a background song using an SKAudioNode and it works fine initially, but when I pause and play the game within a few seconds, the music does not begin playing again. I have tried lots of things like removing the SKAudioNode when the game is paused and adding it again when the game is resumed, but nothing has worked. I have posted a snippet of my code below keeping it as relevant as possible:
class GameScene: SKScene, SKPhysicsContactDelegate {
var backgroundMusic = SKAudioNode(fileNamed: "bg.mp3")
let pauseImage = SKTexture(imageNamed: "pause.png")
var pauseButton:SKSpriteNode!
let playImage = SKTexture(imageNamed: "play2.png")
var playButton:SKSpriteNode!
override func didMoveToView(view: SKView) {
self.addChild(backgroundMusic)
// create pause button
pauseButton = SKSpriteNode(texture: pauseImage)
pauseButton.position = CGPoint(x: self.size.width - pauseButton.size.width, y: pauseButton.size.height)
pauseButton.zPosition = 1
pauseButton.name = "pauseButton"
self.addChild(pauseButton)
// create play button
playButton = SKSpriteNode(texture: playImage)
playButton.position = CGPoint(x: self.size.width - playButton.size.width, y: -playButton.size.height)
playButton.zPosition = 1
playButton.name = "playButton"
self.addChild(playButton)
}
override func touchesBegan(touches: Set<UITouch>, withEvent event: UIEvent?) {
for touch in touches {
if pauseButton.containsPoint(touch.locationInNode(self)) {
let blockOne = SKAction.runBlock({
self.pauseButton.position.y = - self.pauseButton.size.height
self.playButton.position.y = self.playButton.size.height
})
let blockTwo = SKAction.runBlock({
self.view?.paused = true
})
self.runAction(SKAction.sequence([blockOne, blockTwo]))
}
else if(playButton.containsPoint(touch.locationInNode(self))) {
self.playButton.position.y = -self.playButton.size.height
self.pauseButton.position.y = self.pauseButton.size.height
self.view?.paused = false
}
}
}
}
You'll want to look into using AVAudioPlayer and NSNotificationCenter to pass data around the game.
Start the background audio player in your actual GameViewController class.
It's better to do it this way then use SKAudioNode... That's more for sounds that are like sound effects relating to something that happened in gameplay.
By using AVAudioPlayer, the advantage is when the music is paused it's still cued up to play in it's previous spot.
This is one of the few things that will be running regardless of what's going on. So we put it in the GameViewController.
So here's an example of GameViewController code we'd need to start
import AVFoundation
var bgMusicPlayer:AVAudioPlayer?
Then in the GameViewController we make these functions as such
override func viewDidLoad() {
super.viewDidLoad()
// PlayBackgroundSound , PauseBackgroundSound will be your code to send from other "scenes" to use the audio player //
// NSNotificationCenter to pass data throughout the game //
NSNotificationCenter.defaultCenter().addObserver(self, selector: #selector(GameViewController.playBackgroundSound(_:)), name: "PlayBackgroundSound", object: nil)
NSNotificationCenter.defaultCenter().addObserver(self, selector: #selector(GameViewController.pauseBackgroundSound), name: "PauseBackgroundSound", object: nil)
}
func playBackgroundSound(notification: NSNotification) {
let name = notification.userInfo!["fileToPlay"] as! String
if (bgSoundPlayer != nil){
bgSoundPlayer!.stop()
bgSoundPlayer = nil
}
if (name != ""){
let fileURL:NSURL = NSBundle.mainBundle().URLForResource(name, withExtension: "mp3")!
do {
bgSoundPlayer = try AVAudioPlayer(contentsOfURL: fileURL)
} catch _{
bgSoundPlayer = nil
}
bgSoundPlayer!.volume = 1
bgSoundPlayer!.numberOfLoops = -1
// -1 will loop it forever //
bgSoundPlayer!.prepareToPlay()
bgSoundPlayer!.play()
}
}
func pauseBackgroundSound() {
if (bgSoundPlayer != nil){
bgSoundPlayer!.pause()
}
}
Then when you want to use the audio player in your pause or resume button functions.
Remember you need to have the player used in each scene.
import AVFoundation.AVAudioSession
override func didMoveToView(view: SKView) {
try! AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryAmbient)
}
Then if you want to pause or play something just use NSNotificationCenter.
// To Pause in a scene use this line of code in the function you need to pause the music
NSNotificationCenter.defaultCenter().postNotificationName("PauseBackgroundSound", object: self)
/// To Play initially or a new song .. use this line of code in the function you need to play the music
NSNotificationCenter.defaultCenter().postNotificationName("PlayBackgroundSound", object: self, userInfo: "FILE NAME OF BACKGROUND MUSIC")
Whenever I go back to my home screen it replays the music over the already playing music. I tried making an if statements that 'obviously' doesn't work (because I barely know any swift!). Here is my home screen code:
var myAudioPlayer = AVAudioPlayer()
override func viewDidLoad() {
let myFilePathString =
NSBundle.mainBundle().pathForResource("16 March of the Resistance", ofType: "m4a")
if let myFilePathString = myFilePathString
{
let myFilePathURL = NSURL(fileURLWithPath: myFilePathString)
do{
try myAudioPlayer = AVAudioPlayer(contentsOfURL: myFilePathURL)
myAudioPlayer.play()
}catch
{
print("error")
}
}
}
How can I stop it from playing on top of itself?
Here is a picture of my storyboard
enter image description here
The music player is on StartScreen, but when I click the back button, it starts another music player over the current one.
I need code which says
is myAudioPlayer playing?
if yes do not play song again
else play "16 March of the resistance"
enter image description here
You need to check if you are already playing. You can do this by the playing property of the AVAudioPlayer class.
Guard your viewDidLoad with say:
if !myAudioPlayer.playing
Note that I don't know any Swift.
I'm not sure what you mean by "my home screen", is it the device or your main VC? Why are you overriding viewDidLoad()? A previous player could never be playing in the viewDidLoad() call, unless for a leak which is what's needing fixing.
Otherwise, you just need to declare your player as an optional, rather than creating it in the declaration. As for the replaying or overlapping, just check if you player exists, although you wouldnt do this in your viewDidLoad(). ViewDidLoad() is called when the view of the VC is created and the view loaded, which doesn't relate to your lifecycle of the audio.
Depending on your requirements, you might want to create the player somewhere else that is plays throughout the app, rather than restarting when visiting that VC. Otherwise the code below will avoid the duplication. The problem was this var myAudioPlayer = AVAudioPlayer() which would duplicate players and one wasn't being released.
var myAudioPlayer: AVAudioPlayer?
func viewDidLoad() {
// this should never be needed with correct player setup
if let player = myAudioPlayer where player.playing {
return
}
if let pathString = NSBundle.mainBundle().pathForResource("16 March of the Resistance", ofType: "m4a")
{
let url = NSURL(fileURLWithPath: pathString)
do{
try myAudioPlayer = AVAudioPlayer(contentsOfURL: url)
myAudioPlayer?.play()
} catch {
print("error")
}
}
}
Update
So there is no need to make the VC do this. Create a Class of e.g. AudioController, which could even be a singleton. Initialise it in your AppDelegate and trigger this player setup code after initialisation. Your view controllers dont need to know about the audio controller. IF e.g. you do want to modify the volume from a setting page, you just access the singleton object and set its volume.
This way you keep all your audio controller code nicely separated. Dont just think of functionality, think of overall software architecture, with objects having clear responsibilities. Keep your ViewControllers light, they shouldnt be doing much processing.
Take a look at this Singletons , its really easy to setup a singleton in Swift nowadays.
class AudioController {
static let sharedInstance = AudioController()
var audioPlayer: AVAudioPlayer?
let kVolumeKey = "VolumeKey"
let kHasSavedInitialVolumeKey = "HasSavedInitialVolumeKey"
var volume: Float = 0.5
func setup() {
self.loadVolume()
self.setupPlayer()
}
func updatePlayerVolume(volume: Float) {
self.audioPlayer?.volume = volume
self.volume = volume
self.saveVolume()
}
func saveVolume() {
NSUserDefaults.standardUserDefaults().setBool(true, forKey: kHasSavedInitialVolumeKey)
NSUserDefaults.standardUserDefaults().setFloat(self.volume, forKey: kVolumeKey)
NSUserDefaults.standardUserDefaults().synchronize()
}
func loadVolume() {
if NSUserDefaults.standardUserDefaults().boolForKey(kHasSavedInitialVolumeKey) {
self.volume = NSUserDefaults.standardUserDefaults().floatForKey(kVolumeKey)
}
}
func setupPlayer() {
if let pathString = NSBundle.mainBundle().pathForResource("16 March of the Resistance", ofType: "m4a")
{
let url = NSURL(fileURLWithPath: pathString)
do{
try audioPlayer = AVAudioPlayer(contentsOfURL: url)
audioPlayer?.volume = self.volume
audioPlayer?.play()
} catch {
print("error")
}
}
}
}
So, you can just setup the player from your AppDelegate's didFinishLaunching
AudioController.sharedInstance.setupPlayer()
And then you can access the AudioController using AudioController.sharedInstance.whatever when you need to get or set. Also, you will need to implement the players delegate methods for handling the end of the song, possibly triggering a new song to play from a playlist(create a new object for this) or whatever. Your player delegate code is also nicely separated now.
Looking at your storyboard it seems that when you click the Back button on the second ViewController you allocate another start screen and you push it on the screen.
It seems that you're using a UINavigationController if so when the back button is tapped you should call:
self.navigationController?.popViewControllerAnimated(true)
If your showing it as a modal you should dismiss is like this:
self.dismissViewControllerAnimated(true, completion: nil)
I am attempting to present a sheet configuration view (AddSoundEffect) for my main window/view controller (I'm using storyboards), and when the configuration view controller is dismissed, take the values entered in the AddSoundEffect view and pass that back to the main view. My current code in the main view controller:
presentViewControllerAsSheet(self.storyboard!.instantiateControllerWithIdentifier("AddSoundEffect") as! AddSoundViewController
And in the AddSoundViewController.swift file, the code to dismiss it is:
self.dismissViewController(self)
To pass the data, I have a class-independent tuple that I save data to. How do I add a completion handler to presentViewControllerAsSheet, and (optionally) is there a better way to pass the data between view controllers?
Setup: Xcode version 6.4, OS X 10.10.4
Delegation pattern is the easiest way for you.
// Replace this with your tuple or whatever data represents your sound effect
struct SoundEffect {}
protocol AddSoundViewControllerDelegate: class {
func soundViewController(controller: AddSoundViewController, didAddSoundEffect: SoundEffect)
}
//
// Let's say this controller is a modal view controller for adding new sound effects
//
class AddSoundViewController: UIViewController {
weak var delegate: AddSoundViewControllerDelegate?
func done(sender: AnyObject) {
// Dummy sound effect info, replace it with your own data
let soundEffect = SoundEffect()
//
// Call it whenever you would like to inform presenting view controller
// about added sound effect (in case of Done, Add, ... button tapped, do not call it
// when user taps on Cancel to just dismiss AddSoundViewController)
//
self.delegate?.soundViewController(self, didAddSoundEffect: soundEffect)
// Dismiss self
self.dismissViewControllerAnimated(true, completion: {})
}
}
//
// Let's say this controller is main view controller, which contains list of all sound effects,
// with button to add new sound effect via AddSoundViewController
//
class SoundEffectsViewController: UIViewController, AddSoundViewControllerDelegate {
func presentAddSoundEffectController(sender: AnyObject) {
if let addSoundController = self.storyboard?.instantiateViewControllerWithIdentifier("AddSoundEffect") as? AddSoundViewController {
addSoundController.delegate = self
self.presentViewController(addSoundController, animated: true, completion: {})
}
}
func soundViewController(controller: AddSoundViewController, didAddSoundEffect: SoundEffect) {
// This method is called only when new sound effect is added
}
}
Another way is to use closures:
// Replace this with your tuple or whatever data represents your sound effect
struct SoundEffect {}
//
// Let's say this controller is a modal view controller for adding new sound effects
//
class AddSoundViewController: UIViewController {
var completionHandler: ((SoundEffect) -> ())?
func done(sender: AnyObject) {
// Dummy sound effect info, replace it with your own data
let soundEffect = SoundEffect()
//
// Call it whenever you would like to inform presenting view controller
// about added sound effect (in case of Done, Add, ... button tapped, do not call it
// when user taps on Cancel to just dismiss AddSoundViewController)
//
self.completionHandler?(soundEffect)
// Dismiss self
self.dismissViewControllerAnimated(true, completion: {})
}
}
//
// Let's say this controller is main view controller, which contains list of all sound effects,
// with button to add new sound effect via AddSoundViewController
//
class SoundEffectsViewController: UIViewController {
func presentAddSoundEffectController(sender: AnyObject) {
if let addSoundController = self.storyboard?.instantiateViewControllerWithIdentifier("AddSoundEffect") as? AddSoundViewController {
addSoundController.completionHandler = { [weak self] (soundEffect) -> () in
// Called when new sound effect is added
}
self.presentViewController(addSoundController, animated: true, completion: {})
}
}
}
Or many other ways like sending notification, ... Whatever suits your needs. But delegation pattern or closures is the best way to go in this specific case.
I missed that your question is about NSViewController. This example is for iOS, but same pattern can be used on OS X without any issues.
The easiest way to detect sheet opening or closing is to use the Sheet Notifications:
class ViewController: NSViewController, NSWindowDelegate {
override func viewDidLoad(){
NSApplication.sharedApplication().windows.first?.delegate = self
}
func windowDidEndSheet(notification: NSNotification) {
}
func windowWillBeginSheet(notification: NSNotification) {
}
}
I have a NSWindow, on which i apply this:
window.styleMask = window.styleMask | NSFullSizeContentViewWindowMask
window.titleVisibility = NSWindowTitleVisibility.Hidden;
window.titlebarAppearsTransparent = true;
I then add a NSView behind the titlebar to simulate a bigger one.
Now it looks like this:
I want to be able to move the window, by dragging the light-blue view. I have already tried to subclass NSView and always returning true for mouseDownCanMoveWindow using this code:
class LSViewD: NSView {
override var mouseDownCanMoveWindow:Bool {
get {
return true
}
}
}
This didn't work.
After some googling i found this INAppStoreWindow on GitHub. However it doesn't support OS X versions over 10.9, so it's completely useless for me.
Edit1
This is how it looks in the Interface Builder.
How can i move the window, by dragging on this NSView?
None of the answers here worked for me. They all either don't work at all, or make the whole window draggable (note that OP is not asking for this).
Here's how to actually achieve this:
To make a NSView control the window with it's drag events, simply subclass it and override the mouseDown as such:
class WindowDragView: NSView {
override public func mouseDown(with event: NSEvent) {
window?.performDrag(with: event)
}
}
That's it. The mouseDown function will transfer further event tracking to it's parent window.
No need for window masks, isMovableByWindowBackground or mouseDownCanMoveWindow.
Try setting the window's movableByWindowBackground property to true.
There are two ways to do this. The first one would be to set the NSTexturedBackgroundWindowMask as well as the windows background color to the one of your view. This should work.
Otherwise you can take a look at this Sample Code
I somehow managed to solve my problem, i don't really know how, but here are some screenshots.
In the AppDelegate file where i edit the properties of my window, i added an IBOutlet of my contentView. This IBOutlet is a subclass of NSView, in which i've overriden the variable mouseDownCanMoveWindow so it always returns false.
I tried this before in only one file, but it didn't work. This however solved the problem.
Thanks to Ken Thomases and Max for leading me into the right direction.
Swift3.0 Version
override func viewDidAppear() {
//for hide the TitleBar
self.view.window?.styleMask = .borderless
self.view.window?.titlebarAppearsTransparent = true
self.view.window?.titleVisibility = .hidden
//for Window movable with NSView
self.view.window?.isMovableByWindowBackground = true
}
Swift 3:
I needed this but dynamically. It's a little long but well worth it (IMHO).
So I decided to enable this only while the command key is down. This is achieved by registering a local key handler in the delegate:
// MARK:- Local key monitor
var localKeyDownMonitor : Any? = nil
var commandKeyDown : Bool = false {
didSet {
let notif = Notification(name: Notification.Name(rawValue: "commandKeyDown"),
object: NSNumber(booleanLiteral: commandKeyDown))
NotificationCenter.default.post(notif)
}
}
func keyDownMonitor(event: NSEvent) -> Bool {
switch event.modifierFlags.intersection(.deviceIndependentFlagsMask) {
case [.command]:
self.commandKeyDown = true
return true
default:
self.commandKeyDown = false
return false
}
}
which is enabled within the delegate startup:
func applicationDidFinishLaunching(_ aNotification: Notification) {
// Watch local keys for window movenment, etc.
localKeyDownMonitor = NSEvent.addLocalMonitorForEvents(matching: NSEventMask.flagsChanged) { (event) -> NSEvent? in
return self.keyDownMonitor(event: event) ? nil : event
}
}
and its removal
func applicationWillTerminate(_ aNotification: Notification) {
// Forget key down monitoring
NSEvent.removeMonitor(localKeyDownMonitor!)
}
Note that when the commandKeyDown value is changed by the key down handler. This value change is caught by the didset{} to post a notification. This notification is registered by any view you wish to have its window so moved - i.e., in the view delegate
override func viewDidLoad() {
super.viewDidLoad()
// Watch command key changes
NotificationCenter.default.addObserver(
self,
selector: #selector(ViewController.commandKeyDown(_:)),
name: NSNotification.Name(rawValue: "commandKeyDown"),
object: nil)
}
and discarded when the viewWillDisappear() (delegate) or the window controller windowShouldClose(); add this
<your-view>.removeObserver(self, forKeyPath: "commandKeyDown")
So sequence goes like this:
key pressed/release
handler called
notification posted
The view's window isMovableByWindowBackground property is changed by notification - placed within view controller / delegate or where you registered the observer.
internal func commandKeyDown(_ notification : Notification) {
let commandKeyDown : NSNumber = notification.object as! NSNumber
if let window = self.view.window {
window.isMovableByWindowBackground = commandKeyDown.boolValue
Swift.print(String(format: "command %#", commandKeyDown.boolValue ? "v" : "^"))
}
}
Remove the tracer output when happy. See it in action in SimpleViewer on github.