When I used the SharpZipLib to unzip a zip file which has 5000 files on the windows phone 7. It took more than 5 minutes to finish it.
Here is the code:
using (StreamReader httpwebStreamReader = new StreamReader(ea.Result))
{
//open isolated storage to save files
using (IsolatedStorageFile isoStore = IsolatedStorageFile.GetUserStoreForApplication())
{
using (ZipInputStream s = new ZipInputStream(httpwebStreamReader.BaseStream))
{
//s.Password = "123456";//if archive is encrypted
ZipEntry theEntry;
while ((theEntry = s.GetNextEntry()) != null)
{
string directoryName = Path.GetDirectoryName(theEntry.Name);
string fileName = Path.GetFileName(theEntry.Name);
// create directory
if (directoryName.Length > 0)
{
isoStore.CreateDirectory(directoryName);
}
if (fileName != String.Empty)
{
//save file to isolated storage
using (BinaryWriter streamWriter =
new BinaryWriter(new IsolatedStorageFileStream(theEntry.Name,
FileMode.OpenOrCreate, FileAccess.Write, FileShare.Write, isoStore)))
{
int size = 2048;
byte[] data = new byte[2048];
while (true)
{
size = s.Read(data, 0, data.Length);
if (size > 0)
{
streamWriter.Write(data, 0, size);
}
else
{
break;
}
}
}
}
}
}
}
}
Why it's so slow?
How can I speed up the unzip action?
Anyone knows?
I think you need to increase your buffer size. Change the lines
int size = 2048;
byte[] data = new byte[2048];
And change the 2048 to something like 32768 (32*1024).
A 2KB block size is making a lot of individual writes to the flash storage. In my experience that's a somewhat slow thing and can vary from device to device. A 32KB block size should do 16 times fewer but I don't know if that will result in a direct 16x speedup. I'm interested to hear back.
Related
I am trying to upload a .mp4 file, selected from the user's iOS or Android device, to my Azure Media Services account.
This code works for small files ( less than ~95MB):
public static async Task<string> UploadBlob(string blobContainerSasUri, string blobName, byte[] blobContent, string path)
{
string responseString;
int contentLength = blobContent.Length;
string queryString = (new Uri(blobContainerSasUri)).Query;
string blobContainerUri = blobContainerSasUri.Split('?')[0];
string requestUri = string.Format(System.Globalization.CultureInfo.InvariantCulture, "{0}/{1}{2}", blobContainerUri, blobName, queryString);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(requestUri);
request.Method = "PUT";
request.AllowWriteStreamBuffering = false;
request.Headers.Add("x-ms-blob-type", "BlockBlob");
request.ContentLength = contentLength;
request.Timeout = Int32.MaxValue;
request.KeepAlive = true;
int bufferLength = 1048576; //upload 1MB at time, useful for a simple progress bar.
Stream requestStream = request.GetRequestStream();
requestStream.WriteTimeout = Int32.MaxValue;
ProgressViewModel progressViewModel = App.Locator.GetProgressBar(App.Locator.MainViewModel.currentModuleItemId);
MyVideosPage myVideosPage = App.Locator.GetVideosPage(App.Locator.MainViewModel.currentModuleItemId);
FileStream fileStream = new FileStream(path, FileMode.Open, FileAccess.Read);
int nRead = 0;
int currentPos = 0;
while ((nRead = fileStream.Read(blobContent, currentPos, bufferLength)) > 0)
{
await requestStream.WriteAsync(blobContent, currentPos, nRead);
currentPos += nRead;
}
fileStream.Close();
requestStream.Close();
HttpWebResponse objHttpWebResponse = null;
try
{
// this is where it fails for large files
objHttpWebResponse = (HttpWebResponse)request.GetResponse();
Stream responseStream = objHttpWebResponse.GetResponseStream();
StreamReader stream = new StreamReader(responseStream);
responseString = stream.ReadToEnd();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
if (objHttpWebResponse != null)
objHttpWebResponse.Close();
}
return responseString;
}
An exception is thrown after this line is called:
(HttpWebResponse)request.GetResponse();
The exception message is "The request body is too large and exceeds the maximum permissible limit."
The exception StatusCode is "RequestEntityTooLarge".
How can I upload large files? Is this a problem with HttpWebRequest, or Azure Media Services?
Azure Storage supports one shot upload (aka PutBlob API) up to 256MB if you are using the new REST API versions. But since you are not specifying the REST API version, you're defaulting to a very old version where the maximum supported size of one shot upload is 100MB.
Use x-ms-version: 2018-03-28 header to be able to upload up to 256MB in one HTTP request.
If you have to deal with larger files, you will need to use block & commit upload. You will need to use PutBlock API to stage blocks from the source file. Blocks can be up to 100MB each. Then you need to commit all the blocks using the PutBlockList API. If you don't have to deal with this logic yourself, simply use the Azure Storage SDK for .NET (supports Xamarin) and use the uploadFromFile method. It is simple, and resilient.
Inserting smaller images work fine. However, when the image file gets bigger than about 2MB, I get an OutOfMemoryException.
I tried with SqlCeUpdatableRecord and a parameterized SqlCeCommand. Using the SqlCeCommand, the exception is raised at ExecuteNonQuery().
With SqlCeUpdatableRecord, the exception is raised at SqlCeUpdatableRecord.SetBytes().
I’ve tried increasing buffer size and temp file size without it seeming to have an effect. I've debugged with GetTotalMemory and there seems to be plenty of resources available.
Any tips would be highly appreciated.
The SQL Server CE database is synchronized with a main SQL Server database and handling images as separate files would introduce a truckload of other issues. The solution has worked fine for years as most WM handhelds only capture images at 5MP or less. However, newer models with support for 8MP images is causing issues.
Here is a simplified version of the SqlCeUpdateblRecord implementation:
System.Data.SqlServerCe.SqlCeUpdatableRecord newRecord = base.CreateRecord();
newRecord["ImageId"] = ImageId;
newRecord["ImageType"] = ImageType;
//64kb buffer (Have tested with different buffer sizes (up to 8MB)
newRecord.SetBytesFromFile("ImageFull", ImageFullFileName, 65536);
newRecord["ImageThumb"] = ImageThumb;
newRecord["ImageDate"] = ImageDate;
base.Insert(newRecord);
.....
public static void SetBytesFromFile(this SqlCeUpdatableRecord obj, string name, string filename, int buffersize)
{
int _column = obj.GetOrdinal(name);
byte[] buffer = new byte[buffersize];
int bytesread;
long offset = 0;
using (FileStream fs = new FileStream(filename,FileMode.Open))
{
bytesread = fs.Read(buffer, 0, buffersize);
while (bytesread > 0)
{
//This will raise OutOfMemoryException for imagefiles larger than appx. 2mb, regardless of buffersize
obj.SetBytes(_column, offset, buffer, 0, bytesread);
offset = offset + (long)bytesread;
bytesread = fs.Read(buffer, 0, buffersize);
}
}
}
Here is a simplified version of the paramterized query:
using (var cmdInsert = new SqlCeCommand("INSERT INTO [Image_CF] ( [ImageId], [ImageType], [ImageFull], [ImageThumb], [ImageDate]) VALUES " +
" ( #ImageId, #ImageType, #ImageFull, #ImageThumb, #ImageDate)"))
{
cmdInsert.Connection = Database.IrisPdaConnection;
cmdInsert.Parameters.Add("#ImageId", System.Data.SqlDbType.Int).Value = ImageId;
cmdInsert.Parameters.Add("#ImageType", System.Data.SqlDbType.NVarChar).Value = ImageType;
cmdInsert.Parameters.Add("#ImageFull", System.Data.SqlDbType.Image).Value = GetFileAsBytes(ImageFullFileName);
cmdInsert.Parameters.Add("#ImageThumb", System.Data.SqlDbType.Image).Value = ImageThumb;
cmdInsert.Parameters.Add("#ImageDate", System.Data.SqlDbType.NVarChar).Value = ImageDate;
// OutOfMemoryException for images larger than appx. 2MB
cmdInsert.ExecuteNonQuery();
}
public byte[] GetFileAsBytes(string filename)
{
FileStream fs;
byte[] result;
using (fs = new FileStream(filename, FileMode.Open, FileAccess.Read))
{
// a byte array to read the image
result = new byte[fs.Length];
fs.Read(result, 0, System.Convert.ToInt32(fs.Length));
}
return result;
}
I'm trying to save what i have drawn with the pencil as a string , and i do this by SaveAsync() method to put it in an IOutputStream then convert this IOutputStream to a stream using AsStreamForWrite() method from this point things should go fine, however i get a lot of problems after this part , if i use for example this code block:
using (var stream = new MemoryStream())
{
byte[] buffer = new byte[2048]; // read in chunks of 2KB
int bytesRead = (int)size;
while (bytesRead < 0)
{
stream.Write(buffer, 0, bytesRead);
}
byte[] result = stream.ToArray();
// TODO: do something with the result
}
i get this exception
"Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection."
or if i try to convert the stream into an image using InMemoryRandomAccessStream like this:
InMemoryRandomAccessStream ras = new InMemoryRandomAccessStream();
await s.CopyToAsync(ras.AsStreamForWrite());
my InMemoryRandomAccessStream variable is always zero in size.
also tried
StreamReader.ReadToEnd();
but it returns an empty string.
found the answer here :
http://social.msdn.microsoft.com/Forums/windowsapps/en-US/2359f360-832e-4ce5-8315-7f351f2edf6e/stream-inkmanager-strokes-to-string
private async void ReadInk(string base64)
{
if (!string.IsNullOrEmpty(base64))
{
var bytes = Convert.FromBase64String(base64);
using (var inMemoryRAS = new InMemoryRandomAccessStream())
{
await inMemoryRAS.WriteAsync(bytes.AsBuffer());
await inMemoryRAS.FlushAsync();
inMemoryRAS.Seek(0);
await m_InkManager.LoadAsync(inMemoryRAS);
if (m_InkManager.GetStrokes().Count > 0)
{
// You would do whatever you want with the strokes
// RenderStrokes();
}
}
}
}
When i use this function to save image(fetch from net) to IsolatedStorage, i found the file size is larger than that i save from the webbrowse.
public static bool CreateImageFile(string filePath, BitmapImage bitmapImage)
{
//StreamResourceInfo streamResourceInfo = Application.GetResourceStream(new Uri(filePath, UriKind.Relative));
using (isolatedStorage = IsolatedStorageFile.GetUserStoreForApplication())
{
string directoryName = System.IO.Path.GetDirectoryName(filePath);
if (!string.IsNullOrEmpty(directoryName) && !isolatedStorage.DirectoryExists(directoryName))
{
isolatedStorage.CreateDirectory(directoryName);
}
if (isolatedStorage.FileExists(filePath))
{
isolatedStorage.DeleteFile(filePath);
}
//bitmapImage
using (IsolatedStorageFileStream fileStream = isolatedStorage.OpenFile(filePath, FileMode.Create, FileAccess.Write))
{
bitmapImage.CreateOptions = BitmapCreateOptions.None;
WriteableBitmap wb = new WriteableBitmap(bitmapImage);
wb.SaveJpeg(fileStream, wb.PixelWidth, wb.PixelHeight, 0, 100);
fileStream.Close();
}
}
return true;
}
Is that ok to save png images using WriteableBitmap.SaveJpeg(...)?
And is there any function to get the length of BitmapImage?
If it is a PNG, why would you use SaveJpeg to actually store it? Why not simply use the standard "data-to-file" approach? If it is already encoded as a PNG, all you need to do is store the content.
Read more here:
How to: Store Files and Folders for Windows Phone
Writing Data (Windows Phone)
Convert to byte array
byte[] data;
using (MemoryStream ms = new MemoryStream()
{
bitmapImage.SaveJpeg(ms, LoadedPhoto.PixelWidth, LoadedPhoto.PixelHeight, 0, 95);
ms.Seek(0, 0);
data = new byte[ms.Length];
ms.Read(data, 0, data.Length);
ms.Close();
}
Then just get the size of the byte array and convert to something more reasonable (KB, MB...)
In Windows Phone 7 how can I save a BitmapImage to local storage? I need to save the image for caching and reload if it is requested again in the next few days.
If you save the file into IsolatedStorage you can set a relative path to view it from there.
Here's a quick example saving a file that was included in the XAP (as a resource) into Isolated Storage.
using (IsolatedStorageFile isoStore = IsolatedStorageFile.GetUserStoreForApplication())
{
if (!isoStore.FileExists(fileName)
{
var sr = Application.GetResourceStream(new Uri(fileName, UriKind.Relative));
using (var br = new BinaryReader(sr.Stream))
{
byte[] data = br.ReadBytes((int)sr.Stream.Length);
string strBaseDir = string.Empty;
const string DelimStr = "/";
char[] delimiter = DelimStr.ToCharArray();
string[] dirsPath = fileName.Split(delimiter);
// Recreate the directory structure
for (int i = 0; i < dirsPath.Length - 1; i++)
{
strBaseDir = Path.Combine(strBaseDir, dirsPath[i]);
isoStore.CreateDirectory(strBaseDir);
}
using (BinaryWriter bw = new BinaryWriter(isoStore.CreateFile(fileName)))
{
bw.Write(data);
}
}
}
}
You may also be interested in the image caching converters created by Ben Gracewood and Peter Nowaks. They both show saving images into isolated storage and loading them from there.
Another approach I've used is to pass the stream you retrieve for the image in your xap straight into an isolated storage file. Not a lot of moving parts.
using (var isoStore = IsolatedStorageFile.GetUserStoreForApplication()) {
var bi = new BitmapImage();
bi.SetSource(picStreamFromXap);
var wb = new WriteableBitmap(bi);
using (var isoFileStream = isoStore.CreateFile("pic.jpg"))
Extensions.SaveJpeg(wb, isoFileStream, wb.PixelWidth, wb.PixelHeight, 0, 100);
}