first time I use Parallel.For to fill Lists (here ConcurrentBag)
But it seems that the ConcurrentBag receives data from the other threads.
I have a huge amount of data in xSource an the task is to distribute it to several Lists.
What ist wrong here?
public static void getNearXvalues(double deltaX, double xCenter, double[] xSource, out double[] xNear)
{
var xList = new ConcurrentBag<double>();
Parallel.For(0, xSource.Length, i =>
{
if((Mat.Abs(xCenter-xSource)<deltaX)
{
xList.Add(xSource[i]);
}
});
xNear = xList.ToArray();
}
I had the same problem, didn't had time to investigate, I used Semaphore to solve the problem. https://learn.microsoft.com/en-us/dotnet/api/system.threading.semaphore?view=net-5.0
Related
I use the libpd4unity package to communicate with Pure Data. I receive a bang from Pure Data with LibPD.Bang. On a bang event I play sound by FMOD.
Problem is, that I receive bangs frequently, for example once every 500 ms but event doesn't trigger in specific length of frame. Usually length change 1 frame less or more.
Is there a solution for this problem? For example a framerate independent event? I want to know if event (delegate) in Unity3D is framerate independent or not.
Because there is tempo for playing each sound and just 1 frame ruins rhythm.
I need to sync sounds for playing by each separate bang.
Regarding your question on whether delegates are dependent or independent from Unity's framerate, there's no straight answer. It depends on how your delegates are called. Are they called from a thread? Are they executed in a thread?
Coroutines are not framerate independent, they are executed in Unity's loop.
The following script should shine a light on the difference between handling delegates in coroutines and in threads.
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Threading;
public class DelegatesAndFramerate : MonoBehaviour {
delegate void MyDelegate();
MyDelegate myDelegate1; // done with coroutines
MyDelegate myDelegate2; // done with threads
Thread thread;
bool threadDone = false;
private int frameCount = 0;
private int delegate1CallCount = 0;
private int delegate2CallCount = 0;
private int callerLoopsCount_coroutine = 0;
private int callerLoopsCount_thread = 0;
void Start () {
myDelegate1 += Elab1;
myDelegate2 += Elab2;
StartCoroutine(CallerCoroutine());
thread = new Thread(new ThreadStart(CallerThread));
thread.Start();
}
void Update()
{
frameCount++;
}
void Elab1()
{
delegate1CallCount++;
}
void Elab2()
{
delegate2CallCount++;
}
IEnumerator CallerCoroutine()
{
while(true)
{
callerLoopsCount_coroutine++;
myDelegate1();
yield return null;
}
}
void CallerThread()
{
while(!threadDone)
{
callerLoopsCount_thread++;
myDelegate2();
}
}
void OnDestroy()
{
Debug.Log("Frame Count: " + frameCount);
Debug.Log("Delegate Call Count (Coroutine): " + delegate1CallCount);
Debug.Log("Delegate Call Count (Thread): " + delegate2CallCount);
Debug.Log("Caller Loops Count (Coroutine): " + callerLoopsCount_coroutine);
Debug.Log("Caller Loops Count (Thread): " + callerLoopsCount_thread);
threadDone = true;
thread.Join();
}
}
If you attach it to a GameObject and let Unity play for some seconds you'll see that the times the delegate was called from a coroutine is equal to the number of executed frames whilst the times the delegate was called from the thread will be way bigger.
I have experience in interfacing softwares similar to Pure Data and I think what you need is a (rather typical) thread with all your delegates there, create a queue of commands for Unity and digest it in Unity's Update.
Not knowing libPD in the specific this might not be the best practice for the case but it is a widely used approach. Basically the producer-consumer pattern.
Basing on the example GUITextScript.cs, libPD only requires you to subscribe to the right delegates. You don't have control on when these are executed, the library has; so if you keep having this issue it's worth submitting a bug report to the developers I guess.
I try to implement kind of a Videostream that relays on simple JPEG Files.
On my server, a JPEG is being created cyclically by an external Camera.
And I just want to include this Picture in my GWT Application.
My first idea to reload the Picture by a Timer was very simple but not so good: The client opens a connection for each reload-cycle, and the Picture flickers (at least in Firefox).
How could I solve these problems? I was thinking about something like "Web-Sockets", but I don't really know how to do.
I want to avoid a single connection for each reload. My idea was to have something like an open connection that just provides a new Picture as often as the Client asks for.
And how could I avoid the flickering when swapping the Picture?
Any ideas are welcome!
Regards, VanDahlen
A solution to avoid flickering is to have two images absolutely positioned in the same location. A timer would load one or other alternatively in each frame. Set a load handler to each image, so that it changes the z-index when the image is loaded and it restarts the timer.
Adding an extra parameter to the image url, makes the browser ask the server each time to bypass its cache.
If the time between frames is small, normally the browser will re-use the same connection if keep-alive is correctly configured in your server. It normally is enabled with a typical value of 5-15 seconds which you could increase, so if your .jpg images are updated with this periodicity, you don't have to worry and look for a better solution.
I propose a UI solution based on these ideas. But it will work as well if you use a websocket/comet mechanism giving you the last .jpg file in base64 format (just change the url by the value returned).
GWT code:
public void onModuleLoad() {
final Image i1 = new Image();
i1.setWidth("400px");
final Image i2 = new Image();
i2.setWidth("400px");
AbsolutePanel panel = new AbsolutePanel();
panel.add(i1, 0, 0);
panel.add(i2, 0, 0);
panel.setSize("600px", "400px");
RootPanel.get().add(panel);
// You could change this by base64 data if you use comet/websockets
String url = "my_image_url.jpg?";
final Timer loadNext = new Timer() {
boolean b;
int c;
public void run() {
// the counter parameter forces to load the next frame instead of using cache
if (b = !b) {
i1.setUrl(url + c++);
} else {
i2.setUrl(url + c++);
}
}
};
i1.addLoadHandler(new LoadHandler() {
public void onLoad(LoadEvent event) {
i1.getElement().getStyle().setZIndex(1);
i2.getElement().getStyle().setZIndex(0);
loadNext.schedule(1000);
}
});
i2.addLoadHandler(new LoadHandler() {
public void onLoad(LoadEvent event) {
i1.getElement().getStyle().setZIndex(0);
i2.getElement().getStyle().setZIndex(1);
loadNext.schedule(1000);
}
});
loadNext.schedule(1000);
}
If you want to use gwtquery, the code is obviously smaller:
// You could change this by base64 data if you use comet/websockets
final String url = "my_image_url.jpg?";
final GQuery images = $("<img/><img/>").appendTo(document);
images.css($$("position: fixed, top: 10px, left: 600px, width: 400px"));
final Timer timer = new Timer() {
int c;
public void run() {
images.eq(c%2).attr("src", url + c++);
}
};
images.bind("load", new Function(){
public void f() {
$(this).css($$("z-index: 1")).siblings("img").css($$("z-index: 0"));
timer.schedule(1000);
}
});
timer.schedule(1000);
I am trying to analyze what problem i might be having with unsafe threading in my code.
In my mvc3 webapplication i try to the following:
// Caching code
public static class CacheExtensions
{
public static T GetOrStore<T>(this Cache cache, string key, Func<T> generator)
{
var result = cache[key];
if(result == null)
{
result = generator();
lock(sync) {
cache[key] = result;
}
}
return (T)result;
}
}
Using the caching like this:
// Using the cached stuff
public class SectionViewData
{
public IEnumerable<Product> Products {get;set;}
public IEnumerable<SomethingElse> SomethingElse {get;set;}
}
private void Testing()
{
var cachedSection = HttpContext.Current.Cache.GetOrStore("Some Key", 0 => GetSectionViewData());
// Threading problem?
foreach(var product in cachedSection.Products)
{
DosomestuffwithProduct...
}
}
private SectionViewData GetSectionViewData()
{
SectionViewData viewData = new SectionViewData();
viewData.Products = CreateProductList();
viewData.SomethingElse = CreateSomethingElse();
return viewData;
}
Could i run inte problem with the IEnumerable? I dont have much experience with threading problems. The cachedSection would not get touched if some other thread adds a new value to cache right? To me this would work!
Should i cache Products and SomethingElse indivually? Would that be better than caching the whole SectionViewData??
Threading is hard;
In your GetOrStore method, the get/generator sequence is entirely unsynchronized, so any nymber of threads can get null from the cache and run the generator function at the same time. This may - or may not - be a problem.
Your lock statement only locks the setter of cache[string], which is already thread safe and doesn't need to be "extra locked".
The variation of double-checked locking in the cache is suspect, I'd try to get rid of it. Since the thread that never enters the lock() section can get result without a memory barrier, result may not be entirely constructed by the time the thread gets it.
Enumerating the cached IEnumrators is safe as long as nothing modifies them at the same time. If GetSectionViewData() returns an object with immutable (as in non changing) collections, you're safe.
Your code is missing parts like how would Products be populated? Only in GetSectionViewData?
If so, then I don't see a major problem with your code.
There is however a chance that two threads generate the same data(CachedSection) for the same key, it shouldn't create a threading problem except that you are doing the work twice, so if this was an expensive operation I would change the code so it only generates it once per key. If it is not expensive, it works fine as is.
IEnumerable for Products is not touched (assuming you create it separately per thread, but the enumerator on the cache is modified for each insert operation, hence it is not thread safe. So if you are using this I would be careful about that.
How can I convert System.IObservable<string> to System.IObservable<System.Collections.Generic.List<string>>?
public static IObservable<List<string>> GetUrlList(Uri url) {
var result = (from request in Observable.Return(GetWebRequest(url, false))
from response in Observable.FromAsyncPattern<WebResponse>(
request.BeginGetResponse, request.EndGetResponse)()
from item in GetUrlCollection(response).ToObservable()
select item);
return result;
}
Here the type of the item is "list of string". I need to convert that to IObservable List of strings. How can I achieve this?
Try this:
public static IObservable<List<string>> GetUrlList(Uri url)
{
var result = (
from request in Observable.Return(
GetWebRequest(url, false))
from response in Observable.FromAsyncPattern<WebResponse>(
request.BeginGetResponse, request.EndGetResponse)()
from item in GetUrlCollection(response)
.ToObservable()
.ToArray()
select item
.ToList());
return result;
}
My only concern with this whole approach is that your GetUrlCollection(response) is returning an enumerable. You really should code this to return an observable.
Hmmm I think Observable.Start is your friend here. You have a bunch of code that it looks like you are forcing into Observable Sequences, when they really don't look like they are.
Remember Rx is designed to work with sequences of push data. You seem to have a sequence of 1 that happens to be a List. TPL/Task/async would be a good fit here.
If you do want to use Rx, I would suggest avoiding boucing around between IEnumable and IObservable. Doing so is a quick way to creating nasty race conditions and confusing the next developer.
public static IObservable<List<string>> GetUrlList(Uri url)
{
return Observable.Start(()=>
{
var request = GetWebRequest(url, false);
return GetUrlCollection(request);//Code change here??
});
}
Here you can happily be synchronous in your Observable.Start delegate. This should be a lot easier for the next guy to understand (i.e. this is a single value sequence with the UrlCollection as the value).
Should FIFO queue be synchronized if there is only one reader and one writer?
What do you mean by "synchronized"? If your reader & writer are in separate threads, you want the FIFO to handle the concurrency "correctly", including such details as:
proper use of FIFO API should never cause data structures to be corrupted
proper use of FIFO API should not cause deadlock (although there should be a mechanism for a reader to wait until there is something to read)
the objects read from the FIFO should be the same objects, in the same order, written to the FIFO (there shouldn't be missing objects or rearranged order)
there should be a bounded time (one would hope!) between when the writer puts something into the FIFO, and when it is available to the reader.
In the Java world there's a good book on this, Java Concurrency In Practice. There are multiple ways to implement a FIFO that handles concurrency correctly. The simplest implementations are blocking, more complex ones use non-blocking algorithms based on compare-and-swap instructions found on most processors these days.
Yes, if the reader and writer interact with the FIFO queue from different threads.
Depending on implementation, but most likely. You don't want reader to read partially written data.
Yes, unless its documentation explicitly says otherwise.
(It is possible to implement a specialized FIFO that doesn't need synchronization if there is only one reader and one writer thread, e.g. on Windows using InterlockedXXX functions.)
Try this code for concurrent fifo usage:
public class MyObjectQueue {
private static final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
private static final ReadLock readLock;
private static final WriteLock writeLock;
private static final LinkedList<MyObject> objects;
static {
readLock = lock.readLock();
writeLock = lock.writeLock();
objects = new LinkedList<MyObject>();
}
public static boolean put(MyObject p) {
writeLock.lock();
try {
objects.push(p);
return objects.contains(p);
} finally {
writeLock.unlock();
}
}
public static boolean remove(MyObject p) {
writeLock.lock();
try {
return objects.remove(p);
} finally {
writeLock.unlock();
}
}
public static boolean contains(MyObject p) {
readLock.lock();
try {
return objects.contains(p);
} finally {
readLock.unlock();
}
}
public MyObject get() {
MyObject o = null;
writeLock.lock();
try {
o = objects.getLast();
} catch (NoSuchElementException nse) {
//list is empty
} finally {
writeLock.unlock();
}
return o;
}
}