|
Edhy Rijo
|
|
|
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
|
I have a process that will load a couple of million records in BO to do a duplicate record search. My process will import thousands (360K+) of records and then I need to make sure those records does not exist in the database, so instead of querying the data 360k+, I load a BO with all possible duplicate records to check against and this is the million records BO. Now the problem is that even when I do a BO.Clear to remove those records the application keeps using a lot of memory (over 1gb) and that memory is not released until the whole application is closed, so when the import process starts again, it will blow up with a not enough memory to complete the process. So what should I do in order to make sure that the memory used once the million record BO is filled is released when using BO.Clear? Also I am open to suggestions to change my approach if I need to. Thanks!
Edhy Rijo
|
|
|
|
|
Edhy Rijo
|
|
|
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
|
Hi Keith, Thanks for the reference, I downloaded the trial and so far discover that my forms are not being removed from the MDI control in the main/parent form and even when I close any child form, there will be an instance there. I am reviewing the code to see how to get rid of the form instance that should not be there.
Edhy Rijo
|
|
|
|
|
Edhy Rijo
|
|
|
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
|
Hi Trent, In this application I am following the same code as in StrataFlix to launch forms and create the _MDIClient objct. After using ANTS Memory Profiler I noticed that if I open any form, and then close it, a reference of this form will still be hanging. See MovieMaintenance.png picture. The same is happening with my forms, except that my form is using a large memory chunk and being left hanging it is causing the "out of memory" exception. See MyFormProfiler.png. In some cases the memory used by this form is over 1GB and when I close it and open again, the memory used will just keep increasing. I have tried several things to get rid of the hanging form without any luck. Could you please take a look at this in the StrataFlix sample and see if you can come up with a way to really close and dispose the MDI child forms? Thanks!
Edhy Rijo
|
|
|
|
|
Trent Taylor
|
|
|
Group: StrataFrame Developers
Posts: 6.6K,
Visits: 7K
|
To All:
We have made some relatively large changes within StrataFrame recently that have not yet been released which will resolve this issue along with other potential memory leaks. Before releasing this build, we are running and testing it to make sure that there are no residual side-effects. It has been running smoothly and effectively in the field for nearly 2 weeks and will be released to the SF community at large next week.
Note: Though many issues will be resolved without any code changes at all, clean development and the release of any handlers or references is required to clean up memory and prevent memory leaks. StrataFrame has been changed to try and clean up things that may not be handled from the developers side to reduce the risk of this happening. However, it is still the developers responsibility to clean up within their code (i.e. if a handler is added make sure it is removed, etc.).
|
|
|
|
|
Edhy Rijo
|
|
|
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
|
Hi Trent, Those are great news. Thanks! Beside those changes, I don't think they cover the situation explained here about using MDI forms with the same logic used in StrataFlix. Right now my project is being beta tested and this particular form dealing with a couple of million of records got stock with out of memory error, normally closing the form should fix the problem, but I noticed that a form reference is still hanging when I close the form and the same thing happens on the StrataFlix project. I have tried many things to fix the problem, but I have not been able to fix it. Could you please take a look at the StrataFlix, using ANT or any other memory profiler and notice that the form reference is left hanging? At this point I am desperate and open to any suggestion. Thanks!
Edhy Rijo
|
|
|
|
|
Trent Taylor
|
|
|
Group: StrataFrame Developers
Posts: 6.6K,
Visits: 7K
|
Edhy, The offending line of code within the StrataFlix sample is this: ''' <summary> ''' Launches a dialog and adds it to the MDI ''' </summary> ''' <param name="dialogType"></param> ''' <remarks></remarks> Private Sub LaunchDialog(ByVal dialogType As System.Type) '-- Establish Locals Dim f As Form = CType(Activator.CreateInstance(dialogType), Form) '-- Add the handlers to the form
AddFormHandlers(f) '-- Set the parent MDI
f.MdiParent = Me '-- Show the dialog
f.Show()
End Sub If you comment out the code that associates the form with the MDI, the form will release. I tested this a number of different ways including removing any handler, threading close, etc. If it was attached to an MDI, it stuck around. Even on a sample dialog outside of StrataFrame, if you do this (and load the dialog as we are here) it is sticking around. I will play with this some more to see if there is a way to get rid of it. In ANTS, it indicates that the propertStore is holding onto the disposed reference....which indicates that this is lower level. I am going to look at .NET through Reflector and see if I can tell what it is doing.
|
|
|
|
|
Edhy Rijo
|
|
|
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
|
Thanks a lot Trent, I really appreciate the time you are taking on this.
Edhy Rijo
|
|
|
|
|
Trent Taylor
|
|
|
Group: StrataFrame Developers
Posts: 6.6K,
Visits: 7K
|
For a further test, I 100% removed StrataFrame from the formula. Attached here is a plain 100% very simple .NET app that uses a parent/child. If you create the child and close it, it doesn't get released. There are no handlers, code, objects, etc. This is as basic as you can get and it won't release. I even used the plain IsMdiContainer property versus adding an MDIClient. This looks like a .NET issue. Now I am curious and will continue to mess with this...but it does prove StrataFrame has nothing to do with it. 
|
|
|
|
|
Trent Taylor
|
|
|
Group: StrataFrame Developers
Posts: 6.6K,
Visits: 7K
|
One more thing worth noting. A reference does stick around, but only one. If you open the dialog 10 times only a single live instance is sticking around [scratching head]. Since there are not any shared or static references, this is a bit strange.
|
|
|
|
|
Trent Taylor
|
|
|
Group: StrataFrame Developers
Posts: 6.6K,
Visits: 7K
|
It does look to be a .NET issue. Once I started looking this up on Google it appears that this is not uncommon. Looking for a resolution, but I would be curious to see if you get the same results on ANTS as I did on the project I posted.
|
|
|
|
|
Edhy Rijo
|
|
|
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
|
Hi Trent, Yes I got the same results in ANTS. During the past weekend I have been looking like crazy in all google findings about this issue and have not been able to find a solution. I know this is not related to SF, but have you have a time to look at this deeper? Thanks again!
Edhy Rijo
|
|
|
|
|
Edhy Rijo
|
|
|
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
|
Here is some information I found on the web about how to overcome the "System.OutOfMemoryException". Apparently, .Net runtime does not let you go up to 2GB by default, and not matter how much memory you have it will give you the Out Of Memory exception if your application passes 1.1GB of memory. That is exactly my case, I am loading some lookup data 3-5 millions records for a process that needs to check on those records. I am importing around 350 thousand records from a CSV file and each record needs to be checked against those 3-5 millions records. First I had a store procedure to do the checking, but that was painfully slow since it had to go to the database thousands of time to do the check. I could not do the check in groups because a decision needs to be done at the time of checking. So loading the lookup data triggered the Out Of memory exception and it has been a nightmare since the last 2 weeks. There is a program named Editbin.exe which can be use to flag the VB.NET assembly to use memory after the 2GB with the /LARGEADDRESSAWARE parameter. I added this code to my project's Post-Build: call "$(DevEnvDir)..\..\VC\vcvarsall.bat" x86
Editbin.exe /LARGEADDRESSAWARE “$(TargetPath)” So far, with this change I am able to run the import process with less out of memory exceptions. I am still getting the error, but not so frequenly now. Monitoring the memory with task manager, I seen it past the 2GB, then drops to 800k and up again, and without leaving the form I have done at least 3 import test and have not seen the out of memory error. My computer have 8GB Ram, so now will test on one of the beta tester computer with less ram and see what happen. I don't like this temporary solution since it does requires the main .exe file to be hacked with EditBin.exe but until I can test the upcoming SF changes that address possible memory leaks in the BOs I have no other choice so far. I am compiling in .Net 2.0 and have not tested yet the 3.5 runtimes to see if there is a difference. Here is copy of the out of memory exception:
OutOfMemoryException
Exception of type 'System.OutOfMemoryException' was thrown.
Source : mscorlib
Stack Trace:
at System.Collections.Generic.Dictionary`2.Initialize(Int32 capacity)
at System.Collections.Generic.Dictionary`2..ctor(Int32 capacity, IEqualityComparer`1 comparer)
at System.Data.DataView.ResetRowViewCache()
at System.Data.DataView.UpdateIndex(Boolean force, Boolean fireEvent)
at System.Data.DataView.UpdateIndex(Boolean force)
at System.Data.DataView.SetIndex2(String newSort, DataViewRowState newRowStates, IFilter newRowFilter, Boolean fireEvent)
at System.Data.DataTable.get_DefaultView()
at MicroFour.StrataFrame.Business.BusinessLayer.ChangeCurrentDataTable(DataTable NewTable, Boolean AcceptChanges, Boolean IsSharedTable)
at MicroFour.StrataFrame.Business.BusinessLayer.FillDataTable(DbCommand CommandToExecute)
at CardTrackingSystem.Business.bizTransactionItemsImport.FillPreviousRecordLookup(Int32 pFK_Transaction, Int32 pFK_Vendor)
at CardTrackingSystem.frmFirstUsageImport.PreAnalyzeRecord_Step1()
at CardTrackingSystem.frmFirstUsageImport.cmdAnalyzeImportedRecords_Click(Object sender, EventArgs e)
at System.Windows.Forms.ToolStripItem.RaiseEvent(Object key, EventArgs e)
at System.Windows.Forms.ToolStripMenuItem.OnClick(EventArgs e)
at System.Windows.Forms.ToolStripItem.HandleClick(EventArgs e)
at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)
at System.Windows.Forms.ToolStripItem.FireEventInteractive(EventArgs e, ToolStripItemEventType met)
at System.Windows.Forms.ToolStripItem.FireEvent(EventArgs e, ToolStripItemEventType met)
at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)
at System.Windows.Forms.ToolStripDropDown.OnMouseUp(MouseEventArgs mea)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
at System.Windows.Forms.ToolStrip.WndProc(Message& m)
at System.Windows.Forms.ToolStripDropDown.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativewindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativewindow.WndProc(Message& m)
at System.Windows.Forms.Nativewindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
Edhy Rijo
|
|
|
|
|
Edhy Rijo
|
|
|
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
|
Hi Trent, Have you have a chance to take a look at this issue with the MDIClient not disposing the MDI child forms?
Edhy Rijo
|
|
|
|
|
Trent Taylor
|
|
|
Group: StrataFrame Developers
Posts: 6.6K,
Visits: 7K
|
A little. As we had discussed, this is something that we reproduced outside of SF. I have played with this for a while and .NET is holding on to the MDI reference. As of yet, I have not found a solution. But I will look at this some more at the first opportunity. I would be curious to see what Microsoft has to say on the issue. I think that we need to send the sample that we have to them and ask them why this continues to hold on to the reference.
But I will still look at this again.
|
|
|
|
|
Edhy Rijo
|
|
|
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
|
|
|
|
|
|
Edhy Rijo
|
|
|
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
|
Hi Trent, Will the new SF release 1.7.0.6 have any effect in this issue with the MDI forms? My trial version of ANTS expired and I cannot test your sample application with new SF assemblies.
Edhy Rijo
|
|
|
|
|
Trent Taylor
|
|
|
Group: StrataFrame Developers
Posts: 6.6K,
Visits: 7K
|
Most likely the two are unrelated since the code you are referring to is not SF specific but rather .NET specific. But I will run a test to see.
|
|
|
|