Here is some information I found on the web about how to overcome the "System.OutOfMemoryException".
Apparently, .Net runtime does not let you go up to 2GB by default, and not matter how much memory you have it will give you the Out Of Memory exception if your application passes 1.1GB of memory.
That is exactly my case, I am loading some lookup data 3-5 millions records for a process that needs to check on those records. I am importing around 350 thousand records from a CSV file and each record needs to be checked against those 3-5 millions records.
First I had a store procedure to do the checking, but that was painfully slow since it had to go to the database thousands of time to do the check. I could not do the check in groups because a decision needs to be done at the time of checking. So loading the lookup data triggered the Out Of memory exception and it has been a nightmare since the last 2 weeks.
There is a program named Editbin.exe which can be use to flag the VB.NET assembly to use memory after the 2GB with the /LARGEADDRESSAWARE parameter.
I added this code to my project's Post-Build:
call "$(DevEnvDir)..\..\VC\vcvarsall.bat" x86
Editbin.exe /LARGEADDRESSAWARE “$(TargetPath)”
So far, with this change I am able to run the import process with less out of memory exceptions. I am still getting the error, but not so frequenly now. Monitoring the memory with task manager, I seen it past the 2GB, then drops to 800k and up again, and without leaving the form I have done at least 3 import test and have not seen the out of memory error.
My computer have 8GB Ram, so now will test on one of the beta tester computer with less ram and see what happen.
I don't like this temporary solution since it does requires the main .exe file to be hacked with EditBin.exe but until I can test the upcoming SF changes that address possible memory leaks in the BOs I have no other choice so far.
I am compiling in .Net 2.0 and have not tested yet the 3.5 runtimes to see if there is a difference.
Here is copy of the out of memory exception:
OutOfMemoryException
Exception of type 'System.OutOfMemoryException' was thrown.
Source : mscorlib
Stack Trace:
at System.Collections.Generic.Dictionary`2.Initialize(Int32 capacity)
at System.Collections.Generic.Dictionary`2..ctor(Int32 capacity, IEqualityComparer`1 comparer)
at System.Data.DataView.ResetRowViewCache()
at System.Data.DataView.UpdateIndex(Boolean force, Boolean fireEvent)
at System.Data.DataView.UpdateIndex(Boolean force)
at System.Data.DataView.SetIndex2(String newSort, DataViewRowState newRowStates, IFilter newRowFilter, Boolean fireEvent)
at System.Data.DataTable.get_DefaultView()
at MicroFour.StrataFrame.Business.BusinessLayer.ChangeCurrentDataTable(DataTable NewTable, Boolean AcceptChanges, Boolean IsSharedTable)
at MicroFour.StrataFrame.Business.BusinessLayer.FillDataTable(DbCommand CommandToExecute)
at CardTrackingSystem.Business.bizTransactionItemsImport.FillPreviousRecordLookup(Int32 pFK_Transaction, Int32 pFK_Vendor)
at CardTrackingSystem.frmFirstUsageImport.PreAnalyzeRecord_Step1()
at CardTrackingSystem.frmFirstUsageImport.cmdAnalyzeImportedRecords_Click(Object sender, EventArgs e)
at System.Windows.Forms.ToolStripItem.RaiseEvent(Object key, EventArgs e)
at System.Windows.Forms.ToolStripMenuItem.OnClick(EventArgs e)
at System.Windows.Forms.ToolStripItem.HandleClick(EventArgs e)
at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)
at System.Windows.Forms.ToolStripItem.FireEventInteractive(EventArgs e, ToolStripItemEventType met)
at System.Windows.Forms.ToolStripItem.FireEvent(EventArgs e, ToolStripItemEventType met)
at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)
at System.Windows.Forms.ToolStripDropDown.OnMouseUp(MouseEventArgs mea)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
at System.Windows.Forms.ToolStrip.WndProc(Message& m)
at System.Windows.Forms.ToolStripDropDown.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativewindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativewindow.WndProc(Message& m)
at System.Windows.Forms.Nativewindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
Edhy Rijo