i have existing old version images like kernel image,filesys image,application images in my NAND flash.
i want to port the new modified kernel or application image on to the NAND flash by replacing the older one.
But in the new images 90% is common to the old images.
so i don't want the entire new image to transfer.
inspite i am thinking of some kind of comaprision between the old and new images and want to send only the difference on to flash memory. so that i can avoid transfering a larger data.
can it be possible ? i need some guidence to do this.
It's certainly possible, however with flash you'll have to take into account the difference between erase sector size and write sector size (typically the erase block is multiple write sectors in size).
This would be very difficult, for two reasons.
The Linux kernel is stored compressed, so a small change can cause all the compression output following that point to be different.
If a modification changes the size of some code, everything stored after that will have to shift forward or back.
In theory, you could create your own way of linking and/or compressing the kernel so that code stays in one place and compression happens in a block-aware way, but that would be a lot of work -- probably not worth it just to save a few minutes of erase/write time during kernel upgrades.
Related
I am working on a LabVIEW project on which I have to process some video (with for example 5000 images of 640*480 pixels, so lot of data to process). Using a for loop I am processing one image at the time so in this side all is okay. But in the other side, I have to store the results to visualise the results of the wished image after the processing. Until now I always worked with array but here LabVIEW has not enough memory to do the job (which is quite normal).
Is there a best way to change my way to deal with the data, using other solution such as cluster, save the image on the local disk, etc?
For information, the processing is quite long (several minutes for only one image) and I don't have to save the result before the user ask for so I am anticipating the case where all the video is processed without saving the result.
Thank you in advance.
How much RAM do you have? Assuming 4 bytes per pixel, 5000 640 x 480 images would take about 6 GB, so if you have 16 GB RAM or more then you might be able to handle this data in RAM as long as you're using 64-bit LabVIEW and you're careful about how memory is allocated - read through VI Memory Usage from the help, for a start.
Alternatively you can look at storing the data on disk in a format where you can access an arbitrary chunk from the file. I haven't used it much myself but HDF5 seems to be the obvious choice - if you're on Windows you can install the LiveHDF5 library from the VI package manager.
Did you consider to store images as files in the system temporary directory and delete it afterwards? Since the processing takes long time per image, it should be easily possible to have a "image queue" of 5 images always loaded into memory (to avoid aby performance drop due to loading from file right before the processing) and rest would sit on the disk.
I'm creating a program that reads pictures (JPG max size about 10Mb per file) from FlashAir as soon as they're taken, display them in a bigger screen for review and saved them to a local folder. It is paramount to reduce the time from the moment the picture is taken until it is displayed to the user and to prevent loss of quality (they are macro pictures). Now, the camera works with JPG, so changing that is not an option for the moment. All the pictures must be saved locally in the maximum possible quality.
I was wondering what would be the best way to achieve this. Since the FlashAir card is in the camera and moves around, the bottleneck will probably be in the wireless transfer (max speed is 54 Mb/s).
The picture could be displayed withing the Java app or sent to a different app for editing, but I want to reduce I/O operations (I don't want to have to re-read the picture once is saved locally to actually display it).
What is the best way to achieve this using pure Java 8 classes?
My test implementation uses ImageIO.read() and ImageIO.write() methods. The problems I have with this approach is that it takes a long time for the picture to be displayed (it is actually read from the saved folder) and the image is re-encoded and compressed, loosing quality compared to the original file that is in the SD Card.
I feel it should be a way to transfer the bytes very efficiently over the network first and run two parallel processes to save the untouched bytes to disk and decode and display the image (image then could potentially be edited and saved to disk to a different location).
I don't need a fully working example. My main concern is what Java 8 I/O classes are best suited for this job and to know if my approach is the best one to achieve the results.
Edit
After some research I was thinking of using ReadableByteChannel to storage the picture bytes in a ByteBuffer and then pass copies of it to two jobs that will run in parallel: the one saving the bytes would use a FileChannel and the one displaying the image would use them to create an ImageIcon.
I don't know if there is a better/recommended approach.
so I'm trying to publish a looping animation in Flash with a large amount of image sequences in it. (around 3000 frames all told). And I'm having problems with the swf using too much memory when it's played.
The swf is about 80mb, but the file uses an excess of 2gb of ram when played. I don't know why that would be. A memory leak?
My understanding is: that Flash will just load all images in a swf into memory unless you dump the memory somehow.Can anyone explain how to do this? Is it possible? I can't seem to find a solution online.
Thanks
SWF file keeps all images in a compressed format. They can be compressed very well but when playing, they must be decompressed so they take much bigger memory space.
How to optimize that depends on you. There is also a possibility that it cannot be optimized - for example, if SWF does automatic decompression of all images on load. You can test it by checking the free memory after swf is loaded but before the animation is accessed. If standard timeline animation takes too much memory, you can try to use your own custom animation. For example, in an ENTER_FRAME loop you can create only one instance of image for the current frame, and all past instances will be removed and garbage collected. Hard to say because a testing is needed.
Thanks for your responses. As I couldn't find an answer to this, I've resorted to using external video. Cheers for your time.
How does a large file, say a 20GB game, fit into memory? I am keeping myself up just thinking about this and I just don't understand it.
It doesn't. It remains on disk and is read in as needed. Windows has a special API for doing this that makes it completely transparent to the programmer - google MemoryMappedFile that describes how to open a file that is too large to hold in RAM.
Games though, may use different techniques, reading in parts of the file that are needed at any given time so for example, all levels might be held in a single file, but the game will open the filew, seek to the right place, and read the data from the file as the player enters a new level.
It is on the disk. Game reads chunks of it that needed in the meantime.
(I was not able to find a clear answer to my question, maybe I used the wrong search term)
I want to record many images from a camera, with no compression or lossless compression, on a not so powerful device with one single solid drive.
After investigating, I have decided that, if any, the compression will be simply png image by image (this is not part of the discussion).
Given these constraints, I want to be able to record at maximum possible frequency from the camera. The bottleneck is the (only one) hard drive speed. I want to use the RAM for queuing, and the few available cores for compressing the images in parallel, so that there's less data to write.
Once the data is compressed, do I get any gain in writing speed if I stream all the bytes in one single file, or, considering that I am working with a solid drive, can I just write one file (let's say about 1 or 2 MB) per image still working at the maximum disk bandwidth? (or very close to it, like >90%)?
I don't know if it matters, this will be done using C++ and its libraries.
My question is "simply" if by writing my output on a single file instead of in many 2MB files I can expect a significant benefit, when working with a solid state drive.
There's a benefit, not a significant one. A file system driver for a solid state drive already knows how to distribute the data of a file across many non-adjacent clusters so doing it yourself doesn't help. Necessary to fit a large file on a drive that already contains files. By breaking it up, you force extra writes to also add the directory entries for those segments.
The type of a solid state drive matters but this is in general already done by the driver to implement "wear-leveling". In other words, intentionally scatter the data across the drive. This avoids wearing out flash memory cells, they have a limited number of times you can write them before they physically wear out and fail. Traditionally only guaranteed at 10,000 writes, they've gotten better. You'll exercise this of course. Notable as well is that flash drives are fast to read but slow to write, that matters in your case.
There's one notable advantage to breaking up the image data into separate files: it is easier to recover from a drive error. Either from a disastrous failure or the drive just filling up to capacity without you stopping in time. You don't lose the entire shot. But inconvenient to whatever program reads the images off the drive, it has to glue them back together. Which is an important design goal as well, if you make it too impractical with a non-standard uncompressed file format or just too slow to transfer or just too inconvenient in general then it will just not get used very often.