OutOfMemoryException when Inserting larger images in SQL Server CE - image

Inserting smaller images work fine. However, when the image file gets bigger than about 2MB, I get an OutOfMemoryException.
I tried with SqlCeUpdatableRecord and a parameterized SqlCeCommand. Using the SqlCeCommand, the exception is raised at ExecuteNonQuery().
With SqlCeUpdatableRecord, the exception is raised at SqlCeUpdatableRecord.SetBytes().
I’ve tried increasing buffer size and temp file size without it seeming to have an effect. I've debugged with GetTotalMemory and there seems to be plenty of resources available.
Any tips would be highly appreciated.
The SQL Server CE database is synchronized with a main SQL Server database and handling images as separate files would introduce a truckload of other issues. The solution has worked fine for years as most WM handhelds only capture images at 5MP or less. However, newer models with support for 8MP images is causing issues.
Here is a simplified version of the SqlCeUpdateblRecord implementation:
System.Data.SqlServerCe.SqlCeUpdatableRecord newRecord = base.CreateRecord();
newRecord["ImageId"] = ImageId;
newRecord["ImageType"] = ImageType;
//64kb buffer (Have tested with different buffer sizes (up to 8MB)
newRecord.SetBytesFromFile("ImageFull", ImageFullFileName, 65536);
newRecord["ImageThumb"] = ImageThumb;
newRecord["ImageDate"] = ImageDate;
base.Insert(newRecord);
.....
public static void SetBytesFromFile(this SqlCeUpdatableRecord obj, string name, string filename, int buffersize)
{
int _column = obj.GetOrdinal(name);
byte[] buffer = new byte[buffersize];
int bytesread;
long offset = 0;
using (FileStream fs = new FileStream(filename,FileMode.Open))
{
bytesread = fs.Read(buffer, 0, buffersize);
while (bytesread > 0)
{
//This will raise OutOfMemoryException for imagefiles larger than appx. 2mb, regardless of buffersize
obj.SetBytes(_column, offset, buffer, 0, bytesread);
offset = offset + (long)bytesread;
bytesread = fs.Read(buffer, 0, buffersize);
}
}
}
Here is a simplified version of the paramterized query:
using (var cmdInsert = new SqlCeCommand("INSERT INTO [Image_CF] ( [ImageId], [ImageType], [ImageFull], [ImageThumb], [ImageDate]) VALUES " +
" ( #ImageId, #ImageType, #ImageFull, #ImageThumb, #ImageDate)"))
{
cmdInsert.Connection = Database.IrisPdaConnection;
cmdInsert.Parameters.Add("#ImageId", System.Data.SqlDbType.Int).Value = ImageId;
cmdInsert.Parameters.Add("#ImageType", System.Data.SqlDbType.NVarChar).Value = ImageType;
cmdInsert.Parameters.Add("#ImageFull", System.Data.SqlDbType.Image).Value = GetFileAsBytes(ImageFullFileName);
cmdInsert.Parameters.Add("#ImageThumb", System.Data.SqlDbType.Image).Value = ImageThumb;
cmdInsert.Parameters.Add("#ImageDate", System.Data.SqlDbType.NVarChar).Value = ImageDate;
// OutOfMemoryException for images larger than appx. 2MB
cmdInsert.ExecuteNonQuery();
}
public byte[] GetFileAsBytes(string filename)
{
FileStream fs;
byte[] result;
using (fs = new FileStream(filename, FileMode.Open, FileAccess.Read))
{
// a byte array to read the image
result = new byte[fs.Length];
fs.Read(result, 0, System.Convert.ToInt32(fs.Length));
}
return result;
}

Related

Windows 8.1 store xaml save InkManager in a string

I'm trying to save what i have drawn with the pencil as a string , and i do this by SaveAsync() method to put it in an IOutputStream then convert this IOutputStream to a stream using AsStreamForWrite() method from this point things should go fine, however i get a lot of problems after this part , if i use for example this code block:
using (var stream = new MemoryStream())
{
byte[] buffer = new byte[2048]; // read in chunks of 2KB
int bytesRead = (int)size;
while (bytesRead < 0)
{
stream.Write(buffer, 0, bytesRead);
}
byte[] result = stream.ToArray();
// TODO: do something with the result
}
i get this exception
"Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection."
or if i try to convert the stream into an image using InMemoryRandomAccessStream like this:
InMemoryRandomAccessStream ras = new InMemoryRandomAccessStream();
await s.CopyToAsync(ras.AsStreamForWrite());
my InMemoryRandomAccessStream variable is always zero in size.
also tried
StreamReader.ReadToEnd();
but it returns an empty string.
found the answer here :
http://social.msdn.microsoft.com/Forums/windowsapps/en-US/2359f360-832e-4ce5-8315-7f351f2edf6e/stream-inkmanager-strokes-to-string
private async void ReadInk(string base64)
{
if (!string.IsNullOrEmpty(base64))
{
var bytes = Convert.FromBase64String(base64);
using (var inMemoryRAS = new InMemoryRandomAccessStream())
{
await inMemoryRAS.WriteAsync(bytes.AsBuffer());
await inMemoryRAS.FlushAsync();
inMemoryRAS.Seek(0);
await m_InkManager.LoadAsync(inMemoryRAS);
if (m_InkManager.GetStrokes().Count > 0)
{
// You would do whatever you want with the strokes
// RenderStrokes();
}
}
}
}

How to speed up the unzip action on windows phone 7?

When I used the SharpZipLib to unzip a zip file which has 5000 files on the windows phone 7. It took more than 5 minutes to finish it.
Here is the code:
using (StreamReader httpwebStreamReader = new StreamReader(ea.Result))
{
//open isolated storage to save files
using (IsolatedStorageFile isoStore = IsolatedStorageFile.GetUserStoreForApplication())
{
using (ZipInputStream s = new ZipInputStream(httpwebStreamReader.BaseStream))
{
//s.Password = "123456";//if archive is encrypted
ZipEntry theEntry;
while ((theEntry = s.GetNextEntry()) != null)
{
string directoryName = Path.GetDirectoryName(theEntry.Name);
string fileName = Path.GetFileName(theEntry.Name);
// create directory
if (directoryName.Length > 0)
{
isoStore.CreateDirectory(directoryName);
}
if (fileName != String.Empty)
{
//save file to isolated storage
using (BinaryWriter streamWriter =
new BinaryWriter(new IsolatedStorageFileStream(theEntry.Name,
FileMode.OpenOrCreate, FileAccess.Write, FileShare.Write, isoStore)))
{
int size = 2048;
byte[] data = new byte[2048];
while (true)
{
size = s.Read(data, 0, data.Length);
if (size > 0)
{
streamWriter.Write(data, 0, size);
}
else
{
break;
}
}
}
}
}
}
}
}
Why it's so slow?
How can I speed up the unzip action?
Anyone knows?
I think you need to increase your buffer size. Change the lines
int size = 2048;
byte[] data = new byte[2048];
And change the 2048 to something like 32768 (32*1024).
A 2KB block size is making a lot of individual writes to the flash storage. In my experience that's a somewhat slow thing and can vary from device to device. A 32KB block size should do 16 times fewer but I don't know if that will result in a direct 16x speedup. I'm interested to hear back.

Posting file on Background Agent / HttpWebRequest stream buffer keeps growing?

I need to POST a 5MB file from within a ResourceIntensiveTask, where the OS sets a max memory usage of 5MB.
So trying to stream the file directly from storage, but the Stream associated to the HttpWebRequest keeps growing in size. This is the code:
public void writeStream(Stream writer, string filesource, string filename)
{
var store = System.IO.IsolatedStorage.IsolatedStorageFile.GetUserStoreForApplication();
var f = store.OpenFile(filesource, FileMode.Open, FileAccess.Read);
store.Dispose();
byte[] buffer = Encoding.UTF8.GetBytes(String.Format(#"Content-Disposition: form-data; name=""file""; filename=""{0}""\n", filename));
writer.Write(buffer, 0, buffer.Length);
buffer = Encoding.UTF8.GetBytes("Content-Type: application/octet-stream\n");
writer.Write(buffer, 0, buffer.Length);
long initialMemory = Microsoft.Phone.Info.DeviceStatus.ApplicationCurrentMemoryUsage;
buffer = new byte[2048];
int DataRead = 0;
do
{
DataRead = f.Read(buffer, 0, 2048);
if (DataRead > 0)
{
writer.Write(buffer, 0, DataRead);
Array.Clear(buffer, 0, 2048);
}
} while (DataRead > 0);
double increasedMemory = ((double)Microsoft.Phone.Info.DeviceStatus.ApplicationCurrentMemoryUsage - initialMemory) / 1000000;
buffer = Encoding.UTF8.GetBytes("\n--" + boundary + "\n--");
writer.Write(buffer, 0, buffer.Length);
writer.Flush();
}
increasedMemory debug variable is used to get the differential memory before and after the file is read and streamed to the HttpWebRequest, and it gives almost the exact size of the file (5MB) which means the process memory is increasing 5MB.
I am also setting AllowReadStreamBuffering=false to the HttpWebRequest.
How to keep memory low? How to upload large files when memory usage limit is 5MB?
The problem is that without being able to turn off write buffering, the connection to the server is not even made until BeginGetResponse() is called after closing the request stream (verified with WireShark).
The only way I can think of to get around this would be to use sockets directly (although that will be way more complicated if using an SSL connection).
This code works for me and doesn't increase memory usage while sending data to the server. I haven't tested it in a background task but don't see any reason it wouldn't work.
Socket _socket;
const int BUFFERSIZE = 4096;
byte[] writebuffer = new byte[BUFFERSIZE];
string hostName = "www.testdomain.com";
string hostPath = "/test/testupload.aspx";
IsolatedStorageFileStream isoFile;
public void SocketPOST(string hostName, string filesource)
{
using (IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication())
{
if (store.FileExists(filesource))
{
isoFile = store.OpenFile(filesource, FileMode.Open, FileAccess.Read);
}
}
_socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
_socket.SetNetworkRequirement(NetworkSelectionCharacteristics.NonCellular);
SocketAsyncEventArgs socketEventArg = new SocketAsyncEventArgs();
socketEventArg.RemoteEndPoint = new DnsEndPoint(hostName, 80);
socketEventArg.Completed += new EventHandler<SocketAsyncEventArgs>(Socket_Completed);
_socket.ConnectAsync(socketEventArg);
}
private void Socket_Completed(object sender, SocketAsyncEventArgs e)
{
if (e.SocketError == SocketError.Success)
{
switch (e.LastOperation)
{
case SocketAsyncOperation.Connect: // Connected so started sending data, headers first
if (e.ConnectSocket.Connected)
{
StringBuilder sbHeaders = new StringBuilder("POST " + hostPath + " HTTP/1.1\r\n");
sbHeaders.Append("HOST: " + hostName + "\r\n");
sbHeaders.Append("USER-AGENT: MyWP7App/1.0\r\n");
sbHeaders.Append("Content-Type: text/plain; charset=\"utf-8\"\r\n");
sbHeaders.Append("Content-Length: " + isoFile.Length.ToString() + "\r\n\r\n");
byte[] headerBuffer = Encoding.UTF8.GetBytes(sbHeaders.ToString());
e.SetBuffer(headerBuffer, 0, headerBuffer.Length);
if (!e.ConnectSocket.SendAsync(e)) Socket_Completed(e.ConnectSocket, e);
}
break;
case SocketAsyncOperation.Send:
case SocketAsyncOperation.SendTo: // Previous buffer sent so send next one if stream not finished
Array.Clear(writebuffer, 0, BUFFERSIZE);
int DataRead = 0;
DataRead = isoFile.Read(writebuffer, 0, BUFFERSIZE);
if (DataRead > 0)
{
e.SetBuffer(writebuffer, 0, DataRead);
if (!_socket.SendAsync(e)) Socket_Completed(e.ConnectSocket, e);
}
else
{
isoFile.Dispose();
if (!_socket.ReceiveAsync(e)) Socket_Completed(e.ConnectSocket, e);
}
break;
case SocketAsyncOperation.Receive:
case SocketAsyncOperation.ReceiveFrom:
if (e.BytesTransferred > 0)
{
string response = Encoding.UTF8.GetString(e.Buffer, e.Offset, e.BytesTransferred).Trim('\0');
// Check response if necessary
e.ConnectSocket.Shutdown(SocketShutdown.Both);
e.ConnectSocket.Dispose();
}
break;
default:
break;
}
}
}
Note: I've left a lot of the error handling out to keep the example short.
SSL Note: Because SSL works at the TCP level and WP7 doesn't currently support SSL sockets (SslStream) you would need to handle the certificate handshake, cipher exchange, etc yourself to set up the SSL connection on the socket and then encrypt everything being sent (and decrypt everything received) with the agreed algorithms. There has been some success using the Bouncy Castle API so that could be possible (see this blog post).
One thing I noticed: you forgot to dispose f!
I personally would use the code like this:
public void writeStream(Stream writer, string filesource, string filename)
{
using (var store = System.IO.IsolatedStorage.IsolatedStorageFile.GetUserStoreForApplication())
{
long initialMemory = Microsoft.Phone.Info.DeviceStatus.ApplicationCurrentMemoryUsage;
using (var f = store.OpenFile(filesource, FileMode.Open, FileAccess.Read))
{
byte[] buffer = Encoding.UTF8.GetBytes(string.Format(#"Content-Disposition: form-data; name=""file""; filename=""{0}""\n", filename));
writer.Write(buffer, 0, buffer.Length);
buffer = Encoding.UTF8.GetBytes("Content-Type: application/octet-stream\n");
writer.Write(buffer, 0, buffer.Length);
buffer = new byte[2048];
int DataRead = 0;
do
{
DataRead = f.Read(buffer, 0, 2048);
if (DataRead > 0)
{
writer.Write(buffer, 0, DataRead);
}
} while (DataRead > 0);
buffer = Encoding.UTF8.GetBytes("\n--" + boundary + "\n--");
writer.Write(buffer, 0, buffer.Length);
writer.Flush();
}
double increasedMemory = ((double)Microsoft.Phone.Info.DeviceStatus.ApplicationCurrentMemoryUsage - initialMemory) / 1000000;
}
}
The boundary var seems to be missing, so a coding error still remains here!

Display static Google Map image in BlackBerry 5.0

I'm having a really interesting problem to solve:
I'm getting a static google map image, with an URL like this.
I've tried several methods to get this information:
Fetching the "remote resource" as a ByteArrayOutputStream, storing the Image in the SD of the Simulator, an so on... but every freaking time I get an IlegalArgumentException.
I always get a 200 http response, and the correct MIME type ("image/png"), but either way: fetching the image and converting it to a Bitmap, or storing the image in the SD and reading it later; I get the same result... the file IS always corrupt.
I really belive its an encoding problem, or the reading method (similar to this one):
public static Bitmap downloadImage(InputStream inStream){
byte[] buffer = new byte[256];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while (inStream.read(buffer) != -1){
baos.write(buffer);
}
baos.flush();
baos.close();
byte[] imageData = baos.toByteArray();
Bitmap bi = Bitmap.createBitmapFromBytes(imageData, 0, imageData.length, 1);
//Bitmap bi = Bitmap.createBitmapFromBytes(imageData, 0, -1, 1);
return bi;
}
The only thing that comes to mind is the imageData.lenght (in the response, the content length is: 6005 ), but I really can't figure this one out.
Any help is more than welcome...
try this way:
InputStream input = httpConn.openInputStream();
byte[] xmlBytes = new byte[256];
int len = 0;
int size = 0;
StringBuffer raw = new StringBuffer();
while (-1 != (len = input.read(xmlBytes)))
{
raw.append(new String(xmlBytes, 0, len));
size += len;
}
value = raw.toString();
byte[] dataArray = value.getBytes();
EncodedImage bitmap;
bitmap = EncodedImage.createEncodedImage(dataArray, 0,dataArray.length);
final Bitmap googleImage = bitmap.getBitmap();
Swati's answer is good. This same thing can be accomplished with many fewer lines of code:
InputStream input = httpConn.openInputStream();
byte[] dataArray = net.rim.device.api.io.IOUtilities.streamToBytes(input);
Bitmap googleImage = Bitmap.createBitmapFromBytes(dataArray, 0, -1, 1);

Blackberry - ListField with images from filesystem

I use the following code to retrieve image from the phone or SDCard and I use that image in to my ListField. It gives the output but it takes very Long time to produce the screen. How to solve this problem ?? Can any one help me?? Thanks in advance!!!
String text = fileholder.getFileName();
try{
String path="file:///"+fileholder.getPath()+text;
//path=”file:///SDCard/BlackBerry/pictures/image.bmp”
InputStream inputStream = null;
//Get File Connection
FileConnection fileConnection = (FileConnection) Connector.open(path);
inputStream = fileConnection.openInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int j = 0;
while((j=inputStream.read()) != -1) {
baos.write(j);
}
byte data[] = baos.toByteArray();
inputStream.close();
fileConnection.close();
//Encode and Resize image
EncodedImage eImage = EncodedImage.createEncodedImage(data,0,data.length);
int scaleFactorX = Fixed32.div(Fixed32.toFP(eImage.getWidth()),
Fixed32.toFP(180));
int scaleFactorY = Fixed32.div(Fixed32.toFP(eImage.getHeight()),
Fixed32.toFP(180));
eImage=eImage.scaleImage32(scaleFactorX, scaleFactorY);
Bitmap bitmapImage = eImage.getBitmap();
graphics.drawBitmap(0, y+1, 40, 40,bitmapImage, 0, 0);
graphics.drawText(text, 25, y,0,width);
}
catch(Exception e){}
You should read files once (on App start or before screen open, maybe put a progress dialog there), put images in array and use this array in paint.

Resources