I am using processing to play image sequences (amongst other stuff) and have an issue in that when the image sequences are too big or I have too many loading at sketch startup I run out of memory (even after tweaking settings). If there is a way round the above where I can load everything on sketch startup that would be ideal however I have come to the conclusion this is not possible meaning I will have to load items before I used them.
This takes some time (around 8 seconds) and I would like to play a loading video or similar whilst this is going on.
Is there anyway to do this? At the moment the whole sketch just freezes while the app cycles the for loop loading the new image sequence and then continues. Starting a video just before does not work as it simple freezes because Draw() is no longer looping.
Tiny bit of my code below.
//loadRed
void loadRed() {
for (int i = 0; i < numFrames; i += 2) {
String imageName = "f1red"+ nf(i, 4) + ".jpg";
images[i] = loadImage(imageName);
println("Loading - " + imageName);
}
}
Any help is appreciated. Will
Have found a solution to this so going to link here hopefully might help somebody out in the future!
http://processing.org/tutorials/data/
Scroll down to Threads and this will allow asynchronous functions to run.
Hope this helps someone. Will
Related
Recently I encounter an issue while profiling my JS code and found out a very small function in the code which took 3-10ms. After investigation, I found out the function Element.innerHTML is the one who took almost all the CPU time when called 1-2 times inside this function.
For example: element1.innerHTML = "<span>text</span>".
I am searching for an alternative to reduce CPU time.
If you can ignore using Element.innerHTML please do. it is super expensive. Instead, use document.createElement and Element.appendChild instead it.
I converted this code element1.innerHTML = "<span>text</span>" to this code:
var element2 = doc.createElement('span');
element2.innerText = "text";
element1.appendChild(element2);
When I did this change in both places I dropped the CPU from each innerHTML usage from 1.5ms to 0.01ms, and even 0ms in most cases.
Then I have a need to add NO_BREAK sign (" ") in different locations that existed in the old code but all the suggestions I got were to use innerHTML for this sign, which will add the CPU time I want to prevent.
The solution to this is using:
element.appendChild(document.createTextNode('\u00A0'))
which will do the same effect without innerHTML. Even using Element.insertAdjacentHTML is parsing the HTML and will take more CPU time, so I tried to ignore the use of it as well.
I hope it helps.
I am a complete novice! Apologies if this appears utterly basic.
I have a distribution which changes over time (cycles) and I want to show this (single series) on a dynamic plot. I have achieved it by recording a macro while I change the yvalues (by 1 row at a time) and then adding in a delay as follows:
Application.Wait (Now + TimeValue("00:00:01"))
However, I don't know how to define the yvalues range such that it will jump to the next row (of y values) and display that.
It could be up to 200 cycles.
I would like to use R3C3 type notation to define the y values and a 'for i < 200.....next i' approach, but I have tried and failed several times.
The specific issue I have is demonstrated here; see the C3:M3 changing up to C7:M7: please see attached image.
Code issue
Any tips? Thanks.
I am trying to run a CNN on the cloud (Google Cloud ML) because my laptop does not have a GPU card.
So I uploaded my data on Google Cloud Storage. A .csv file with 1500 entries, like so:
| label | img_path |
| label_1| /img_1.jpg |
| label_2| /img_2.jpg |
and the corresponding 1500 jpgs.
My input_fn looks like so:
def input_fn(filename,
batch_size,
num_epochs=None,
skip_header_lines=1,
shuffle=False):
filename_queue = tf.train.string_input_producer(filename, num_epochs=num_epochs)
reader = tf.TextLineReader(skip_header_lines=skip_header_lines)
_, row = reader.read(filename_queue)
row = parse_csv(row)
pt = row.pop(-1)
pth = filename.rpartition('/')[0] + pt
img = tf.image.decode_jpeg(tf.read_file(tf.squeeze(pth)), 1)
img = tf.to_float(img) / 255.
img = tf.reshape(img, [IMG_SIZE, IMG_SIZE, 1])
row = tf.concat(row, 0)
if shuffle:
return tf.train.shuffle_batch(
[img, row],
batch_size,
capacity=2000,
min_after_dequeue=2 * batch_size + 1,
num_threads=multiprocessing.cpu_count(),
)
else:
return tf.train.batch([img, row],
batch_size,
allow_smaller_final_batch=True,
num_threads=multiprocessing.cpu_count())
Here is what the full graph looks like (very simple CNN indeed):
Running the training with a batch size of 200, then most of the compute time on my laptop (on my laptop, the data is stored locally) is spent on the gradients node which is what I would expect. The batch node has a compute time of ~12ms.
When I run it on the cloud (scale-tier is BASIC), the batch node takes more than 20s. And the bottleneck seems to be coming from the QueueDequeueUpToV2 subnode according to tensorboard:
Anyone has any clue why this happens? I am pretty sure I am getting something wrong here, so I'd be happy to learn.
Few remarks:
-Changing between batch/shuffle_batch with different min_after_dequeue does not affect.
-When using BASIC_GPU, the batch node is also on the CPU which is normal according to what I read and it takes roughly 13s.
-Adding a time.sleep after queues are started to ensure no starvation also has no effect.
-Compute time is indeed linear in batch_size, so with a batch_size of 50, the compute time would be 4 times smaller than with a batch_size of 200.
Thanks for reading and would be happy to give more details if anyone needs.
Best,
Al
Update:
-Cloud ML instance and Buckets were not in the same region, making them in the same region improved result 4x.
-Creating a .tfrecords file made the batching take 70ms which seems to be acceptable. I used this blog post as a starting point to learn about it, I recommend it.
I hope this will help others to create a fast data input pipeline!
Try converting your images to tfrecord format and read them directly from graph. The way you are doing it, there is no possibility of caching and if your images are small, you are not taking advantage of the high sustained reads from cloud storage. Saving all your jpg images into a tfrecord file or small number of files will help.
Also, make sure your bucket is a single region bucket in a region that had gpus and that you are submitting to cloudml in that region.
I've got the similar problem before. I solved it by changing tf.train.batch() to tf.train.batch_join(). In my experiment, with 64 batch size and 4 GPUs, it took 22 mins by using tf.train.batch() whilst it only took 2 mins by using tf.train.batch_join().
In Tensorflow doc:
If you need more parallelism or shuffling of examples between files, use multiple reader instances using the tf.train.shuffle_batch_join
https://www.tensorflow.org/programmers_guide/reading_data
I'm using Xamarin-Forms-Labs ISoundService in Xamarin to play audio in a Timer.
Every 30 seconds I have it play an audio file (each file is different)
I can get it to play subsequent files if the file size is small (less than 5k) but if it larger is keeps replaying the "larger audio" file in place of the subsequent clips.
Any thoughts how I can resolve this? Am I "stopping" the audio properly. Should I async stop since it is async play?
I appreciate the time and expertise
Xamarin-Forms-Labs ISoundService
https://github.com/XLabs/Xamarin-Forms-Labs/blob/master/src/Platform/XLabs.Platform/Services/Media/ISoundService.cs
My code
var audioFile = intervalSettingsAudioDict[currentInterval];
Console.WriteLine(audioFile);
soundService = DependencyService.Get<ISoundService>();
if (soundService.IsPlaying){
soundService.Stop();
}
soundService.Volume = 1.0;
soundService.PlayAsync(audioFile);
I think the problem is that the default behavior of DependencyService.Get() is to act as a Singleton: http://forums.xamarin.com/discussion/22174/is-a-dependencyservice-implementation-class-treated-as-a-singleton.
I solved it using the following:
var SoundService = DependencyService.Get<ISoundService>(DependencyFetchTarget.NewInstance);
This worked on iOS, but I'm still troubleshooting an unrelated problem on Android.
HTH.
In my Azure web role OnStart() I need to deploy a huge unmanaged program the role depends on. The program is previously compressed into a 400-megabytes .zip archive, splitted to files 20 megabytes each and uploaded to a blob storage container. That program doesn't change - once uploaded it can stay that way for ages.
My code does the following:
CloudBlobContainer container = ... ;
String localPath = ...;
using( FileStream writeStream = new FileStream(
localPath, FileMode.OpenOrCreate, FileAccess.Write ) )
{
for( int i = 0; i < blobNames.Size(); i++ ) {
String blobName = blobNames[i];
container.GetBlobReference( blobName ).DownloadToStream( writeStream );
}
writeStream.Close();
}
It just opens a file, then writes parts into it one by one. Works great, except it takes about 4 minutes when run from a single core (extra small) instance. Which means the average download speed about 1,7 megabytes per second.
This worries me - it seems too slow. Should it be so slow? What am I doing wrong? What could I do instead to solve my problem with deployment?
Adding to what Richard Astbury said: An Extra Small instance has a very small fraction of bandwidth that even a Small gives you. You'll see approx. 5Mbps on an Extra Small, and approx. 100Mbps on a Small (for Small through Extra Large, you'll get approx. 100Mbps per core).
The extra small instance has limited IO performance. Have you tried going for a medium sized instance for comparison?
In some ad-hoc testing I have done in the past I found that there is no discernable difference between downloading 1 large file and trying to download in parallel N smaller files. It turns out that the bandwidth on the NIC is usually the limiting factor no matter what and that a large file will just as easily saturate it as many smaller ones. The reverse is not true, btw. You do benefit by uploading in parallel as opposed to one at a time.
The reason I mention this is that it seems like you should be using 1 large zip file here and something like Bootstrapper. That would be 1 line of code for you to download, unzip, and possibly run. Even better, it won't do it more than once on reboot unless you force it to.
As others have already aptly mentioned, the NIC bandwidth on the XS instances is vastly smaller than even a S instance. You will see much faster downloads by bumping up the VM size slightly.