Filestreaming System.OutofMemoryException

Sharon Chapman

Hi,
I am getting an System.OutofmemoryException when using Filestream to load zip file. the file is 521MB. My code is:

//Insert using Filestream, file into SQL Server Table
        private void btnInsert_Click(object sender, EventArgs e)
        {
            OpenFileDialog openFileDlg = new OpenFileDialog();
            openFileDlg.InitialDirectory = Directory.GetCurrentDirectory();
            if (openFileDlg.ShowDialog() == DialogResult.OK)
            {
                FileInfo fi = new FileInfo(openFileDlg.FileName);
                FileStream fs = new FileStream(fi.FullName, FileMode.Open, FileAccess.Read);
                BinaryReader rdr = new BinaryReader(fs);
                byte[] fileData = rdr.ReadBytes((int)fs.Length);
                rdr.Close();
                fs.Close();

My program dies on this line: byte[] fileData = rdr.ReadBytes((int)fs.Length);
i have also attached the detail for the System.OutofMemoryException. I am runing this on a Windows 7, 64bit

machine.

thanks for your help.

Sharon

3 Replies

Know the answer? Post it — somebody with the same question will find it here.

Vulpes14y ago
If you were loading 28 large files one after the other, then - depending on what other references you have to them - it's possible that you could have had several large byte arrays in memory at the same time which, even in an x64 application, was causing an out of memory error.

If you're only loading one file at a time and then disposing of the byte array, you shouldn't have a problem in an x64 application if you have 8 GB of RAM and the files don't exceed 2 GB individually. The latter is the maximum object size in .NET.

However, if it's an x86 application, I wouldn't like to be 100% sure that the problem couldn't occur again. Such an application is (by default) limited to 2 GB of memory but this includes the memory used by dlls loaded into the same process and, depending on what else you're doing, it's not unusual to run into memory problems when attempting to load arrays of this size. If by studying how much memory your application is using in Task Manager or otherwise, you think that this could occur then it's possible to change the 2 GB application limit to 3 GB. Check this article for more details:

http://msdn.microsoft.com/en-us/library/windows/desktop/bb613473(v=vs.85).aspx
Sharon Chapman14y ago
Hi,

As I mentioned I have a machine that has 8 GB of memory. I had to shut down my machine to take it home last night.

When I tried to load the file again today it loaded with no problem. Normally I will only be loading one file at a

time, but in this case I was testing my Development database and was loading 28 files one right after the other.

Now my question is, since it is working ok now, is there a problem I still need to fix?

Sharon
Vulpes14y ago
The chances are that you really are running out of sufficient memory (or a big enough chunk of contiguous memory) to accomodate a 521 MB file.

You could test that by commenting out your code and seeing if the out of memory error still occurs when you simply try to allocate a byte array of that size.

If you're building a 32-bit application (albeit on a 64-bit OS), then you should be able to improve matters by changing to an x64 build though this may introduce other problems.

Is it really necessary for you to load such a big zip file into memory or if, say, you're just wanting to unzip it, could you launch an external utility using the Process class to do that?