I'm developing a program that will be able to draw the audio waveform using the streamed amplitude data from a microphone or Line-in. The way I thought to do this would be to draw each point from the sample data at a rate equal to the sample rate, going 1 step over in the x direction with each point drawn. Therefore I would need to update the JavaFx Application thread around 44100 times a second to draw each point. Before I start doing this I wanted to test my idea out by just drawing a straight line and only updating each point every half a second. I'm using the Timeline class to do this. My code looks like this:
public class JavaFxPractice extends Application {
private int xValue = 50;
#Override
public void start(Stage primaryStage) {
Pane pane = new Pane();
EventHandler<ActionEvent> eventHandler = e -> {
xValue++;
Line point = new Line(xValue,50,xValue,50);
pane.getChildren().add(point);
};
Timeline animation = new Timeline(new KeyFrame(Duration.millis((500)), eventHandler));
animation.setCycleCount(500);
animation.play();
Scene scene = new Scene(pane, 600, 500);
primaryStage.setTitle("Streaming Test");
primaryStage.setScene(scene);
primaryStage.show();
}
}
However every time I do this my program becomes unresponsive and I have to force close it. I noticed that if I do the same thing but instead make Text blink on and off, it works perfectly fine. Is there a reason Lines aren't able to be drawn using the Timeline class? Does it put too much of a load on the thread? If so, what ways can I go around solving my idea. I just want to be able to draw a waveform in real time, updating at around 44,100 times a second.
I recommend AnimationTimer for any ongoing animation. It attempts to update as close to 60 fps as it can. (I just read some comments following the first one, recommending AnimationTimer. These folks are right. IDK why they didn't post that as an answer themselves.)
The question then becomes what to display. If I were attacking this problem, here is what I would try:
make an array to hold positions, one bucket per pixel, for the size of the display.
make a function that draws the data from such an array, that can be called by the animation timer
make a concurrent-safe queue to hold these arrays (e.g., ConcurrentLinkedQueue)
load the ConcurrentLinkedQueue with reads from your AudioInputLine
poll from the ConcurrentLinkedQueue from the AnimationTimer
To get the timings to work out, you may need to use decimation (e.g., throwing out every 2nd or 2 out of 3 or more PCM data points), or linear interpolation if the needed decimation doesn't come out to an easy-to-use rational fraction. In other words, you are not tied to a 1:1 correspondence between the array (tied to pixels) and the PCM data points. The more decimation you use, the more high frequencies will be lost.
Related
I have a game that requires the player to roll two die. As this is a multiplayer game, the way I currently do this is have 6 animations (1 for each die's outcome). When the player clicks a button, it sends a request to my server code. My server code determines the die's outcome and sends the results to the client. The client then plays the corresponding animations.
This works ok, but has some issues. For instance, if the server sends back two of the same values (two 6's, for example) then the animations don't work correctly. As both animations are the same, they overlay each other, and it looks like only one die was rolled.
Is there a better way to do this? Instead of animations, using "real" dice? If that's the case, I always need to be sure to "pre-determine" the outcome of the dice roll, on the server. I also need to make sure the dice don't fall off the table or jostle any of the other player pieces on the board.
thanks for any ideas.
The server only needs to care about the value result, not running physics calculations.
Set up 12 different rolling animations:
Six for the first die
Six for the second die
Each one should always end with the same modeled face pointing upwards (the starting position isn't relevant, only the ending position). For the latter steps you'll probably want to adjust the model's UV coordinates to use a very tall or very wide texture (or just a slice of a square one). So not like this but rather all in a line 1-2-3-4-5-6.
The next step is picking a random animation to play. You've already got code to run a given animation, just set it to pick randomly instead of based on the die-roll-value from the server:
int animNum = Mathf.Floor(Random.Next()*6);
Finally, the fun bit. Adjusting the texture so that the desired face shows when the animation is done. I'm going to assume that you arrange your faces along the top edge of your square texture. Material.SetTextureOffset().
int showFace = Mathf.Floor(Random.Next()*6); //this value should come from the server
die.renderer.material.SetTextureOffset(1f/6 * showFace,0);
This will set the texture offset such that the desired face will show on top. You'll even be able to see it changing in the inspector. Because of the UVs being arranged such that each face uses the next chunk over and because textures will wrap around when reaching the edge (unless the texture is set to Clamp in its import settings: you don't want this here).
Note that this will cause a new material instance to be instantiated (which is not very performant). If you want to avoid this, you'll have to use a material property block instead.
You could simulate the physics on the server, keep track of the positions and the orientations of the dice for the duration of the animation, and then send the data over to the client. I understand it's a lot of data for something so simple, but that's one way you can get the rolls to appear realistic and synced between all clients.
If only Unity's physics was deterministic that would be a whole lot easier.
EDIT: I've opted for the second approach as I got 150+ fps even when all 3 tile layers fill the entire screen.
EDIT 2: I read a lot about vertex buffer objects and how they would be great for static geometry and although I still have no idea how to turn my 2D tiles into a VBO and store it on the GPU memory, it definitely seems like the way to go if anyone else is looking for a fast way to render static geometry/quads.
I'm making a game like Super Meat Boy and was wondering if it would be better/faster to store level tiles in an array list and do a camera bounds overlap test to see if it should be rendered.
foreach(Tile tile in world.tiles) {
if(Overlap(camera.bounds, tile))
render(tile);
}
Or would a 2D array storing every grid square and only reading off between camera bounds be better?
int left = (int)(camera.position.x - camera.width/2);
int right = (int)(camera.position.x + camera.width/2) + 1;
int top = (int)(camera.position.y - camera.height/2); // WHY XNA DO YOU UPSIDE DOWN!!!
int bottom = (int)(camera.position.y + camera.width/2) + 1;
for(int x = left; x < right; x++) {
for(int y = top; y < bottom; y++) {
render(world.tiles[x][y]);
}
}
The camera can fit 64*36 tiles on screen which is 2300 odd tiles to read off using the latter approach but is doing an overlap test with every tile in the level any better? I read an answer about joining matching adjacent tiles into a larger quad and just repeating the texture (although I'm using a texture atlas so I'm not sure how to repeat a region on a texture).
Cheers guys.
As per my past experience I can share the details. In 2D map, normally map is like 0 - N long. Now N is far longer then screen size. Now at first I tried to loading everything at once. But it is bit of a overhead. Ok, it was like very much of a overhead. And I endup with 0 FPS. As I want different kind of object for me. So, even repeating same object and saving memory is not working. Then I tried bounding things with reference to screen. So, objects are there and but they are not getting rendered. So, it is moved from away from draw pipe line. And game back to life.
Now, for further performance with C# 4.0 I can use TPL and async and await with draw. It is like better version of threading. So, you can throw stuff there and let it be render at will.
Here is deal wiht XNA or any kinda graphics library. There is complete graphics rendering pipeline. And that makes things whole lot slow. Specifically if PC is old and only have 64MB graphics card to support only wide screen. Your game will be deployed to any kinda machine right??!!
So, if I explain in language of XNA, update is simple code and run it as fast as it can, there is nothing to stop it. But draw is has complete pipe line ahead of it. And that is sole reason of having begin and end. So, after end it can start pushing things to pipe line. [Here] (http://classes.soe.ucsc.edu/cmps020/Winter11/readings/hlsl.pdf) article for reference.
So, here is a deal rendering pipeline is needed but there is no need that is should be slow and blocking. Just make it multi-threaded and things will quite faster for you. If you want more terse then you have to use C# at it fullest including Linked list and stuff. But that will be like last stage.
I hope I have given enough details to provide you an answer. Please let me know if any further details needed.
this is an algorithm/data structure question about making different animations at the same time. For example, a ball is falling down one pixel in a millisecond, a bullet is moving 5 pixels in a ms, and a man is moving 1 pixel in 20 milliseconds. And think that there are hundreds of them together. What is the best way of putting all animations together, moving what we need to move in one function call, and removing the ones whose animation is completed? I don't want to create a thread for each one. What I want to do is to create one thread moving all items and sleeping until an object needs to be moved.
Note: I'm using Java/Swing, printing objects and images in JPanel.
I recently did something similar in Python. I don't know if this is the best method, but here's what I did.
Create an abstract Event class with the following public interface:
tick - calculates how much time has passed since the last tick. Perform work proportional to that time span. This should be called frequently to create the illusion of smooth movement; maybe sixteen times a second or so.
isDone - returns true when the Event has finished occuring.
Make a subclass of Event for anything that takes more than one frame to finish. Rotating, scaling, color changing, etc. You might create a TweenEvent subclass of Event if you want to move an image from one part of the screen to another. During each tick, redraw the image in a position farther away from the original position, and farther towards the destination position.
You can run many Events concurrently, like so:
Array<Event> events = new Array<Event>();
//add a bunch of TweenEvents here - one for a bullet, one for a ball, etc.
while(True){
Sleep(1/16);
for(Event e in events){
e.tick();
if (e.isDone()){events.remove(e);}
}
}
I'm making a 2D sidescrolling space shooter-type game, where I need a background that can be scrolled infintely (it is tiled or wrapped repeatedly). I'd also like to implement parallax scrolling, so perhaps have one lowest background nebula texture that barely moves, a higher one containing far-away stars that barely moves and the highest background containing close stars that moves a lot.
I see from google that I'd have each layer move 50% less than the layer above it, but how do I implement this in libgdx? I have a Camera that can be zoomed in and out, and in the physical 800x480 screen could show anything from 128x128 pixels (a ship) to a huge area of space featuring the textures wrapped multiple times on their edges.
How do I continuosly wrap a smaller texture (say 512x512) as if it were infinitely tiled (for when the camera is zoomed right out), and then how do I layer multiple textures like these, keep them together in a suitable structure (is there one in the libgdx api?) and move them as the player's coords change? I've looked at the javadocs and the examples but can't find anything like this problem, apologies if it's obvious!
Hey I am also making a parrallax background and trying to get it to scroll.
There is a ParallaxTest.java in the repository, it can be found here.
this file is a standalone class, so you will need to incorporate it into your game how you want. and you will need to change the control input since its hooked up to use touch screen/mouse.
this worked for me. as for repeated bg, i havent gotten that far yet, but i think you just need to basic logic as in, ok one screen away from the end, change the first few screens pos to line up at the end.
I have not much more to say regarding to the Parallax Scrolling than PFG already did. There is indeed an example in the repository under the test folder and several explanations around the web. I liked this one.
The matter with the background is really easy to solve. This and other related problems can be approached by using modular algebra. I won't go into the details because once shown is very easy to understand.
Imagine that you want to show a compass in your screen. You have a texture 1024x16 representing the cardinal points. Basically all you have is a strip. Letting aside the considerations about the real orientation and such, you have to render it.
Your viewport is 300x400 for example, and you want 200px of the texture on screen (to make it more interesting). You can render it perfectly with a single region until you reach the position (1024-200) = 824. Once you're in this position clearly there is no more texture. But since it is a compass, it's obvious that once you reach the end of it, it has to start again. So this is the answer. Another texture region will do the trick. The range 825-1023 has to be represented by another region. The second region will have a size of (1024-pos) for every value pos>824 && pos<1024
This code is intended to work as real example of a compass. It's very dirty since it works with relative positions all the time due to the conversion between the range (0-3.6) to (0-1024).
spriteBatch.begin();
if (compassorientation<0)
compassorientation = (float) (3.6 - compassorientation%3.6);
else
compassorientation = (float) (compassorientation % 3.6);
if ( compassorientation < ((float)(1024-200)/1024*3.6)){
compass1.setRegion((int)(compassorientation/3.6*1024), 0, 200, 16);
spriteBatch.draw(compass1, 0, (Gdx.graphics.getHeight()/2) -(-250 + compass1.getTexture().getHeight()* (float)1.2), Gdx.graphics.getWidth(), 32 * (float)1.2);
}
else if (compassorientation > ((float)(1024-200)/1024*3.6)) {
compass1.setRegion((int)(compassorientation/3.6*1024), 0, 1024 - (int)(compassorientation/3.6*1024), 16);
spriteBatch.draw(compass1, 0, (Gdx.graphics.getHeight()/2) -(-250 + compass1.getTexture().getHeight()* (float)1.2), compass1.getRegionWidth()/200f * Gdx.graphics.getWidth() , 32 * (float)1.2);
compass2.setRegion(0, 0, 200 - compass1.getRegionWidth(), 16);
spriteBatch.draw(compass2, compass1.getRegionWidth()/200f * Gdx.graphics.getWidth() , (Gdx.graphics.getHeight()/2) -(-250 + compass1.getTexture().getHeight()* (float)1.2), Gdx.graphics.getWidth() - (compass1.getRegionWidth()/200f * Gdx.graphics.getWidth()) , 32 * (float)1.2);
}
spriteBatch.end();
You can use setWrap function like below:
Texture texture = new Texture(Gdx.files.internal("images/background.png"));
texture.setWrap(Texture.TextureWrap.Repeat, Texture.TextureWrap.Repeat);
It will draw background repeatedly! Hope this help!
Beneath where you initialize your Texture for the object. Then beneath that type in this
YourTexture.setWrap(Texture.TextureWrap.Repeat, Texture.TextureWrap.Repeat);
Where YourTexture is your texture that you want to parallax scroll.
In Your render file type in this code.
batch.draw(YourTexture,0, 0, 0 , srcy, Gdx.graphics.getWidth(),
Gdx.graphics.getHeight());
srcy +=10;
It is going to give you an error so make a variable called srcy. It is nothing too fancy.
Int srcy
I want to write a paint program in the style of MS Paint.
For painting things on screen when the user moves the mouse, I have to wait for mouse move events and draw on the screen whenever I receive one. Apparently, mose move events are not sent very often, so I have to interpolate the mouse movement by drawing a line between the current mouse position and the previous one. In pseudocode, this looks something like this:
var positionOld = null
def handleMouseMove(positionNew):
if mouse.button.down:
if positionOld == null:
positionOld = positionNew
screen.draw.line(positionOld,positionNew)
positionOld = positionNew
Now my question: interpolating with straight line segments looks too jagged for my taste, can you recommend a better interpolation method? What method do GIMP or Adobe Photoshop implement?
Alternatively, is there a way to increase the frequency of the mouse move events that I receive? The GUI framework I'm using is wxWidgets.
GUI framework: wxWidgets.
(Programming language: Haskell, but that's irrelevant here)
EDIT: Clarification: I want something that looks smoother than straight line segments, see the picture (original size):
EDIT2: The code I'm using looks like this:
-- create bitmap and derive drawing context
im <- imageCreateSized (sy 800 600)
bitmap <- bitmapCreateFromImage im (-1) -- wxBitmap
dc <- memoryDCCreate -- wxMemoryDC
memoryDCSelectObject dc bitmap
...
-- handle mouse move
onMouse ... sw (MouseLeftDrag posNew _) = do
...
line dc posOld posNew [color := white
, penJoin := JoinRound
, penWidth := 2]
repaint sw -- a wxScrolledWindow
-- handle paint event
onPaint ... = do
...
-- draw bitmap on the wxScrolledWindow
drawBitmap dc_sw bitmap pointZero False []
which might make a difference. Maybe my choices of wx-classes is why I'm getting a rather low frequency of mouse move events.
Live demos
version 1 - more smooth, but more changing while you draw: http://jsfiddle.net/Ub7RV/1/
version 2 - less smooth but more stable: http://jsfiddle.net/Ub7RV/2/
The way to go is
Spline interpolation of the points
The solution is to store coordinates of the points and then perform spline interpolation.
I took the solution demonstrated here and modified it. They computed the spline after you stop drawing. I modified the code so that it draws immediately. You might see though that the spline is changing during the drawing. For real application, you probably will need two canvases - one with the old drawings and the other with just the current drawing, that will change constantly until your mouse stops.
Version 1 uses spline simplification - deletes points that are close to the line - which results in smoother splines but produce less "stable" result. Version 2 uses all points on the line and produces much more stable solution though (and computationally less expensive).
You can make them really smooth using splines:
http://freespace.virgin.net/hugo.elias/graphics/x_bezier.htm
But you'll have to delay the drawing of each line segment until one frame later, so that you have the start and end points, plus the next and previous points available for the calculation.
so, as I see the problem of jagged edge of freehand made curve, when the mouse are moved very fast, is not solved !!! In my opinion there are need to work around with the polling frequency of mousemove event in the system i.e. using different mouse driver or smf.. And the second way is the math.. using some kind of algorithm, to accuratly bend the straight line between two points when the mouse event is polled out.. For clear view you can compare how is drawed free hand line in photoshop and how in mspaint.. thanks folks.. ;)
I think you need to look into the Device Context documentation for wxWidgets.
I have some code that draws like this:
//screenArea is a wxStaticBitmap
int startx, starty;
void OnMouseDown(wxMouseEvent& event)
{
screenArea->CaptureMouse();
xstart = event.GetX();
ystart = event.GetY();
event.Skip();
}
void OnMouseMove(wxMouseEvent& event)
{
if(event.Dragging() && event.LeftIsDown())
{
wxClientDC dc(screenArea);
dc.SetPen(*wxBLACK_PEN);
dc.DrawLine(startx, starty, event.GetX(), event.GetY());
}
startx = event.GetX();
starty = event.GetY();
event.Skip();
}
I know it's C++ but you said the language was irrelevant, so I hope it helps anyway.
This lets me do this:
which seems significantly smoother than your example.
Interpolating mouse movements with line segments is fine, GIMP does it that way, too, as the following screenshot from a very fast mouse movement shows:
So, smoothness comes from a high frequency of mouse move events. WxWidgets can do that, as the example code for a related question demonstrates.
The problem is in your code, Heinrich. Namely, drawing into a large bitmap first and then copying the whole bitmap to the screen is not cheap! To estimate how efficient you need to be, compare your problem to video games: a smooth rate of 30 mouse move events per second correspond to 30fps. Copying a double buffer is no problem for modern machines, but WxHaskell is likely not optimized for video games, so it's not surprising that you experience some jitter.
The solution is to draw only as much as necessary, i.e. just the lines, directly on the screen, for example as shown in the link above.
I agree with harviz - the problem isn't solved. It should be solved on the operating system level by recording mouse movements in a priority thread, but no operating system I know of does that. However, the app developer can also work around this operating system limitation by interpolating better than linear.
Since mouse movement events don't always come fast enough, linear interpolation isn't always enough.
I experimented a little bit with the spline idea brought up by Rocketmagnet.
Instead of putting a line between two points A and D, look at the point P preceding A and use a cubic spline with the following control points B = A + v' and C = D - w', where
v = A - P,
w = D - A,
w' = w / 4 and
v' = v * |w| / |v| / 4.
This means we fall into the second point with the same angle as the line interpolation would, but go out a starting point in the same angle the previous segment came in, making the edge smooth. We use the length of the segment for both control point distances to make the size of the bend fit its proportion.
The following picture shows the result with very few data points (indicated in grey).
The sequence starts at the top left and ends in the middle.
There is still some level of uneasiness here which may be alleviated if one uses both the previous and the next point to adjust for both angles, but that would also mean to draw one point less than what one has got. I find this result already satisfactory, so I didn't try.