I'm adding a subclassed CALayer onto video, and exporting using AVAssetExportSession, but am stumped on getting CABasicAnimation to work for a custom property. I started out with this approach: Core animation callback for progress
There's a "progress" property, that I scale from 0 to 1 using CABasicAnimation to match the timing of the video. On-screen, in a playerView with an AVSynchronizedLayer, "progress" gets updated, and everything is hunky dory. It works exactly as I'd expect.
On export, however, the CABasicAnimation for the progress parameter does not seem to update. I can set different "from" values for the animation, and the "progress" value is set to that for the first frame that gets rendered -- but never changes after that. I've checked the CALayer, as well as the model and presentation layers, and they all have the initial "from" value, each time a frame gets rendered.
If I attach other CABasicAnimation objects to my layer (e.g. scaling, position), they work as expected.
My overall flow goes a bit like this
AVMutableComposition gets created, with audio and video tracks inserted
Build a layer view hierarchy, attaching CABasicAnimation objects. One animation is for the "progress" parameter, and that's where the trouble is.
AVVideoCompositionCoreAnimationTool is created, to connect the video with the CA layers
AVAssetExportSession generates the video
Parameters related to stock animation key paths seem to be fine. The key path for "progress" does not seem to update. I'm suspecting that AVSynchronizedLayer does something for the on-screen updates, and that's not happening for the session export -- but I'm stumped as to what it is that I need to do.
My CALayer subclass implements needsDisplayForKey, and it does get checked for the "progress" key before the render starts -- so I think that the CABasicAnimation, and the rendering pipeline, are aware that "progress" should change.
+ (BOOL)needsDisplayForKey:(NSString*)key {
if ([key isEqualToString:#"progress"])
return YES;
return [super needsDisplayForKey:key];
}
My working theory is that AVVideoCompositionCoreAnimationTool sees the basic animation for my "progress" key path, sets things to the start value at the beginning of a render, and then does not update as the render goes along. And the reason it doesn't update? Because I'm dense, and have totally missed doing something important.
If there's an example along the lines of adding a time code overlay, that would great to look at. I don't think what I'm doing is that exotic, it works great on-screen, but I'm clearly missing some trick with the video export.
--
An update; still no progress on getting CABasicAnimation to work as expected for custom properties, and when exporting video to a file. There seems to be a lot of layer copying/remapping/etc, happening behind the scenes, and I'm sure I'm doing something wrong.
I've got a work-around now that gets the job done, but is a hack. I create an integer frame count property for the view object, and increment it each time the view gets redrawn. The video export is at 30 frames per second, so all I need to do to find out "when" the frame is, is to divide by the frame rate. As far as I can tell, the individual frames are rendered in a "timing perfect" universe, and then compression gets applied, so the actual playback rate may vary.
Related
Is there any API to listen for a particular frame during an animation of a Sprite in CreateJS? There was kind of a way to trigger events at particular frames in a Flash MovieClip, but not sure how to do this in CreateJS.
I could achieve this by listening to the change event of the Sprite while it animates, check the currentFrame on each event, check if the frame index is the one to which I want to react, BUT, this seems rather laboured, and means I have to hard code to an index instead of a frame label.
Interestingly, the _animations property of a Sprite seems to be intended to be private, so although I can use this property to ascertain the length of the animation, again, would still have to hard code to a index value. (Out of curiosity, why not have mySprite.length() API? Flash's MovieClip has a number of frames prop...).
Is there a way to make frame labels in CreateJS? In the docs here, I can see the use of:
instance.gotoAndStop("frameName");
...but I don't see any documentation on how to set frame labels in a SpriteSheet for a particular animation sequence, or how to retrieve the index of the frame associated with a frame label.
Thanks for your help!
The gotoAndStop and gotoAndPlay methods both allow for a frame number or animation name (not frame name). You can label animations when you create a SpriteSheet, but you can not otherwise make named keyframes inside of animations.
To determine the keyframe, you could check the currentAnimationFrame (doc) on your main tick function, or listen for the change event (doc) from the Sprite.
sprite.on("tick", function() {
if (sprite.currentAnimationFrame == 10) {
// do something
}
});
Note that depending on your app framerate, you might have to store the previous frame, and check if the playhead has passed a certain frame, as EaselJS may skip frames depending on the speed of playback.
--
The length property is a good idea, but it would likely have to be a method that you pass in an animation name. I have logged an issue on the EaselJS GitHub
I'm an engineer and we are currently porting our Red5 + Flash game into a Node.js + Easeljs html5 application.
Basicly: it's a board game, not an rpg. The layer system means we have multiple canvasses, based on functionally. For example there is a static background stage, with images. There is a layer for just the timers.
At default, all canvas size is 1920x1080, if needed we downscale to fit to the resolution.
The first approach used kinetic.js, but the performance fallen when the game got complex. Then we switched to easel, because it's abstraction level is lower, so we can decide how to implement some more function, not just use the provided robust one.
I was optimistic, but now it's starting to show slowness again, that's why I want to look deeper inside and do fine performance tuning. (Of course everything is fine in Chrome, Firefox is the problem, but the game must run smoothly on all modern browser).
The main layer (stage) is the map, contains ~30 containers, in each there is a complex custom shape, ~10 images. The containers are listening to mouse events, like mouseover, out, click. Currently, for example on mouseover I refill the shape with gradient.
Somehow, when I use cache, like the way in the tuts the performance get even worse, so I assume I'm messing up something.
I collected some advanced questions:
In the described situation when can I use cache and how? I've already tried cache on init, cacheUpdate after fill with other color or gradient, then stage.update(). No impact.
If I have a static, never changing stage cache doesn't make sense on that layer, right?
What stage.update() exactly do? Triggering the full layer redraw? The doc mentions some kind of intelligent if changed then redraw effect.
If I want to refill a custom shape with new color or gradient I have to completely redraw its graphics, not just use a setFill method, right?
In easel there is no possibility to redraw just a container for example, so how can I manage to not update the whole stage, but just the one container that changed? I thought I can achieve this with caching, cache all containers the just update the one that changed, but this way didn't work at all for me.
Does it make sense to cache bitmap images? If there are custom shapes and images in a container what is better? Cache the container or just the shape in container.
I found a strange bug, or at least an interesting clue. My canvas layers totally overlapping. On the inferior layers the mouseover listening is working well, but the click isn't on the very same container/object.
How can I produce a click event propagation to overlapped layers those have click listeners? I've tried it with simple DOM, jquery, but the event objects were far away from what canvas listeners wanted to get.
In brief, methods and props I've already played with bare success when tried tuning: cache(), updateCache(), update(), mouseEnabled, snapToPixel, clear(), autoClear, enableMouseOver, useRAF, setFPS().
Any answer, suggestion, starting point appreciated.
UPDATE:
This free board game is a strategy game, so you are facing a world map, with ~30 territories. The custom shapes are the territories and a container holds a territory shape and the icons that should be over the territory. This container overlapping is minimal.
An example mouse event is a hover effect. The player navigate over the territory shape then the shape is getting recolored, resized, etc and a bubble showing up with details about the place.
Basically, maximum amount of 1-3 container could change at once (except the init phase -> all at this time). Not just the animations and recoloring slow in FF, but the listener delay is high too.
I wrote a change handler, so I only stage.update() up on tick the modified stages and the stages where an animation is running (tweenjs).
In my first approach I put every image to the container that could be needed at least once during the game, so I only set visible flags on images (not vectors).
Regarding caching:
There are some strange caching-issues, somehow the performance can drop with certain sizes of the caching rectangle: CreateJS / EaselJS Strange Performance with certain size shapes
(2) Depending on how often you call stage.update();
(3)
Each time the update method is called, the stage will tick any
descendants exposing a tick method (ex. BitmapAnimation) and render
its entire display list to the canvas. Any parameters passed to update
will be passed on to any onTick handlers.
=> Afaik it rerenders everything if not cached
(4) Yes.
(5) No. (I don't know of any)
(6) If the content's of the container don't change often, I'd cache the whole container, otherwise the container will be reconstructed every frame.
I have a question though: Why do you use multiple canvases? How many do you use? I could imagine that using multiple canvases might slow down the game.
How many sprites do you use in total?
2: if your layer or stage doesn't change, don't call stage.update() for that layer (so it doesn't gets rerendered, gives me a much lower cpu!)
For example, keep a global "stagechanged" variable and set this to true when something has changed:
createjs.Ticker.addEventListener("tick",
function() {
if (stagechanged)
{
stagechanged = false;
stage.update();
}
});
(or do you already use this, as stated in your "update"?)
4: I found a way to update for example the fill color :)
contaier1.shape1.graphics._fillInstructions[0].params[1] = '#FFFFFF';
(use chrome debugger to look at the _fillInstructions array to see which array position contains your color)
5: I found a way to just paint one container :)
//manual draw 1 component (!)
var a = stage.canvas.getContext("2d");
a.save();
container1.updateContext(a); //set position(x,y) on context
container1.draw(a);
a.restore();
My Cocoa application collects events (instances of NSManagedObject) that need to be displayed in a timeline. My initial approach was to use an existing Javascript based widget (I tried using Simile Timeline and Timeglider) and display the timeline using a WebView control. This works in principle, however unfortunately both these widgets do not handle BC dates very well, which is an important requirement for my app.
The events in my app have date ranges from 500.000BC up to recent dates. Event dates are only expressed with a year. Their day, month and time attributes are irrelevant.
After discarding the Javascript approach, I remain with the option to display the timeline using a custom Cocoa control. As I found none suitable, I will have to develop that myself.
This would be my first custom Cocoa control and after thinking about this for a while I've come up with the following rough design:
I need a custom control to render the actual time line. This control is probably based on an NSView. This control should calculate its size based on the number of tick marks on the time line multiplied by the width (pixels) between each mark. For example the timeline is made up of centuries, each century 100 pixels wide. A time line of events between 10.000BC and 5.000BC would then be 5000 pixels wide (10000 - 5000 = 5000 years, equals 50 centuries).
I need a ScrollView to wrap the timeline to allow it to support scrolling behaviour. There's only need for scrolling horizontally.
I need something to represent an actual event. I'm thinking of using existing controls for this, probably the standard round button and a label wrapped together as a single control.
I need a custom control to render a tick mark on the time line.
Taking this as the basic design for my time line component in Cocoa, would that work or am I completely missing the point?
The basic approach sounds fine.
Apple has a good example of creating a custom NSView called "TreeView". It's a good sample to understand.
https://developer.apple.com/library/mac/#samplecode/TreeView/Introduction/Intro.html#//apple_ref/doc/uid/DTS40010131
“TreeView” presents an example of a creating an entirely new custom
view from scratch (by directly subclassing NSView). Its implementation
illustrates many of the considerations involved in creating a custom
view, including issues of content layout, drawing, handling user
interaction, and providing Accessibility support.
Another thing you may want to consider is zoom in and out. If you have a long timeline, I imagine you may want to zoom out and then zoom in on a cluster of activity. If you have one event in 10k BC and then a cluster of events much later, the user could scroll through tons of empty space trying to find events. Another approach would be to have a mini timeline above that's fit to/static size which is sort of like an index with lines showing activity points - then clicking on that would auto scroll to that point. That may be a nice to have depending on your data.
Some thoughts:
For something this custom drawn, you'll want to override drawRect to draw your lines and layout your subControls.
If you're drawing your background or any part of the views, ensure you enable layer backed views:
[self setWantsLayer:YES];
If you can, as you noted, try leverage existing controls that you add and layout. In my custom controls, I maintained data structures independent of the views/controls that represented the state of all the objects. The in drawRect, I detected the view changing and I called my layoutSubviews function. My layoutSubViews function would do the math from my data structures and create or move the frame of existing controls. That worked well for resize and zooming. If you zoom, your labels ad markers will need to react well to being zoomed really small - perhaps text drops out at some point etc...
if ([self dataSource] &&
!NSEqualRects(_prevRect, [self bounds]))
{
// layoutViews is my custom function that worked over the data structures
// and moved the frame
[self layoutViews];
}
_prevRect = [self bounds];
Hope that helps.
I needed to display some Cocoa widgets on top of an NSOpenGLView in an existing app. I followed the example in Apple's LayerBackedOpenGLView example code. The NSOpenGLView is given a backing layer using:
[glView setWantsLayer:YES]
Then the Cocoa NSView with the widgets is added as a subview of the glView. This is basically working and is twice ad fast as my previous approach where I added the NSView containing the widgets to a child window of the window containing the glView (this was the other solution I found on the web).
There were two problems.
The first is that some textures that I use with blending were no longer getting the blend right. After searching around a bit it looked like I might need to clear the alpha channel of the OpenGLView. This bit of code that I call after drawing a frame seems to have fixed this problem:
Code:
glColorMask(FALSE, FALSE, FALSE, TRUE); //This ensures that only alpha will be effected
glClearColor(0, 0, 0, 1); //alphaValue - Value to which you need to clear
glClear(GL_COLOR_BUFFER_BIT);
glColorMask(TRUE, TRUE, TRUE, TRUE); //Put color mask back to what it was.
Can someone explain why this is needed when using the CALayer, but not without?
The second problem I don't have a solution for. It seems that when I pan to the part of the scene where problem is #1 was observed, the frame rate drops from something like 110 FPS down to 10 FPS. Again, this only started happening after I added the backing layer. This doesn't always happen. Sometimes the FPS stays high when panning over this part of the scene but that is rare. I assume it must have something with how the textures here are blended, but I have no idea what.
Any thoughts?
I did figure out a workaround to the slowdown. The OpenGL view has a HUD (heads up display) view that goes on top of it. I had installed another NSView as a subview if it. Both the HUD and the subview have lots of alpha manipulation and for some reason that tickled a real slowdown in compositing the layers. I could easily instal this subview as a subview of the OpenGL view and when I did this everything sped up again. So although I don't fully understand the slowdown, I do have a good work around for it.
CATransition is quite unusual. Consider the following code.
CATransition* trans=[CATransition animation];
trans.duration=0.5;
trans.type=kCATransitionFade;
[self.holdingView.layer addAnimation:trans forKey:nil];
self.loadingView.hidden=YES;
self.displayView.hidden=NO;
Notice that nowhere did I tell the transition that I wanted to display the displayView rather than loadingView, so the views must somehow access the transition themselves. Can anyone explain in more detail how this works?
When you add the transition as an animation, an implicit CATransaction is begun. From that point on, all modifications to layer properties are going to be animated rather than immediately applied. The way the CATransition performs this animation to to take a snapshot of the view before the layer properties are changed, and a snapshot of what the view will look like after the layer properties are changed. It then uses a filter (on Mac this is Core Image, but on iPhone I'm guessing it's just hard-coded math) to iterate between those two images over time.
This is a key feature of Core Animation. Your draw logic doesn't generally need to deal with the animation. You're given a graphics context, you draw into it, you're done. The system handles compositing that with other images over time (or rotating it in space, or whatever). So in the case of changing the hidden state, the initial-state fully composited image is blended with the final-state composted image. Very fast on a GPU, and it doesn't really matter what change you made to the view.