Bulk Saving?


Author
Message
Edhy Rijo
E
StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)
Group: StrataFrame Users
Posts: 2.4K, Visits: 23K
Hi guys,



I have an import process of CSV files with more than 300 thousand records, I have to read each one and compare against my records and then insert them in another table for future reference with other imports.



My approach is to loop through the imported records collection and then add them to the BO using BO.NewRow(), so far so good, the problem comes when I need to save them using BO.Save(), it takes a long time to actually insert those 300+K records.



I am using the DDT for all my data modeling, and this table is setup to use Store Procedures for all Insert/Update/Delete and also is using an AutoIncrement PK.



In the BO I have turn off the Update/Delete ConcurrencyType and there is no value in the RowVersionOrTimestampColumn.



Is there anything else I may need to do to save those records faster?




Edhy Rijo

Replies
Trent Taylor
Trent Taylor
StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)
Group: StrataFrame Developers
Posts: 6.6K, Visits: 6.9K
Bulk copies....can't live without'em! BigGrin
Edhy Rijo
E
StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)
Group: StrataFrame Users
Posts: 2.4K, Visits: 23K
BTW, is there a way to simply bulk update the records instead of inserting them? Another process is to loop through the inserted records a 2nd or 3th time to re-process them, and after that, the modified ones should be save back to the server.

Edhy Rijo

Edhy Rijo
E
StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)
Group: StrataFrame Users
Posts: 2.4K, Visits: 23K
Trent,



Talking about concurrency, in this same process, when I need to made some modifications to the imported data, I am using an SF BO with UpdateConcurrencyType and DeleteConcurrencyType set to OFF, but when I try to save any modifications I will get the SF Concurrency Manager form but without showing any value to be selected.



In this case I don't want or need any type of concurrency checking, what or where else do I need to check to make sure this SF Concurrency Manager form does not show up and my records are simply just saved?

Edhy Rijo

Trent Taylor
Trent Taylor
StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)
Group: StrataFrame Developers
Posts: 6.6K, Visits: 6.9K
Yes. There are a number of ways to make sure that there is not any concurrency exception popping up, for example, you can handle the ConcurrencyException event and manage this yourself (which you would just ignore any collisions).



Most likely you are doing this on an SF dialog. If so, you can set the AutoHandleCollisions property to false and it should prevent that dialog from appearing.
Edhy Rijo
E
StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)StrataFrame VIP (6.4K reputation)
Group: StrataFrame Users
Posts: 2.4K, Visits: 23K
Thanks for the info.



I believe the AutoHandleCollisions property will handle All or Nothing action, is there a way to control which BO to exclude from the AutoHandleCollisions action?

Edhy Rijo

Trent Taylor
Trent Taylor
StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)StrataFrame Developer (14K reputation)
Group: StrataFrame Developers
Posts: 6.6K, Visits: 6.9K
That is the only place that the dialog is fired from. So if you want to handle this on a BO by BO basis, then you will have to handle the ConcurrencyException event.
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Threaded View
Threaded View
Edhy Rijo - 16 Years Ago
Keith Chisarik - 16 Years Ago
Keith Chisarik - 16 Years Ago
Edhy Rijo - 16 Years Ago
Keith Chisarik - 16 Years Ago
Edhy Rijo - 16 Years Ago
Edhy Rijo - 16 Years Ago
Keith Chisarik - 16 Years Ago
Edhy Rijo - 16 Years Ago
Keith Chisarik - 16 Years Ago
Edhy Rijo - 16 Years Ago
Keith Chisarik - 16 Years Ago
             Bulk copies....can't live without'em! :D
Trent L. Taylor - 16 Years Ago
Edhy Rijo - 16 Years Ago
                     Trent,

Talking about concurrency, in this same process,...
Edhy Rijo - 16 Years Ago
                         Yes. There are a number of ways to make sure that there is not any...
Trent L. Taylor - 16 Years Ago
                             Thanks for the info.

I believe the AutoHandleCollisions...
Edhy Rijo - 16 Years Ago
                                 That is the only place that the dialog is fired from. So if you want...
Trent L. Taylor - 16 Years Ago
ChanKK - 15 Years Ago
Dustin Taylor - 15 Years Ago

Similar Topics

Reading This Topic

Login

Explore
Messages
Mentions
Search