StrataFrame Forum
Back
Login
Login
Home
»
.NET Forums
»
General .NET Discussion
»
Saving big text files in a Varchar Max field?
Saving big text files in a Varchar Max field?
Post Reply
Like
2
Saving big text files in a Varchar Max field?
View
Flat Ascending
Flat Descending
Threaded
Options
Subscribe to topic
Print This Topic
RSS Feed
Goto Topics Forum
Author
Message
Edhy Rijo
E
Edhy Rijo
posted 15 Years Ago
ANSWER
HOT
Topic Details
Share Topic
E
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
Hi guys,
I am currently saving big text files (450,000+ lines) in a Varchar(Max) field, for a while the performance of getting a record with such data decreased and it is noticeable in the browser dialogs and reports.
I am wonder is there is a better way to deal with so much data in a single field, so would it be possible to kind of compress the data in the field somehow?
I know that I may have this field in another table to separate it from the main table and do some magic to make it look as part of the record in the form, but if that is no the best way I am open to any suggestions.
Please keep in mind that currently I am showing the data in the form in a readonly textbox, but I can change the design to not show it up when the form loads and put a button so the user can get the data and then show it, but at least that will not slow down the browser dialog.
Thanks!
Edhy Rijo
Reply
Like
2
Replies
Edhy Rijo
E
Edhy Rijo
posted 15 Years Ago
ANSWER
Post Details
Share Post
E
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
Trent L. Taylor (02/18/2010)
Edhy,
You are aware that you can update a single field as a blob or chunk with events that notify you of the progress, right?
Trent, Dustin,
I was not aware of the Blob field, while working with VFP in the past we always tried to stay away of Blob fields for inestability reasons and it did not cross my mind to even think about them in this case. Since these CSV files are really big, I like the idea of compressing them and be able to show a progress bar during this process, so I will take you on your offer for the sample in VB if possible (I know you are switching to C#
)
Edhy Rijo
Reply
Like
3
Trent Taylor
Trent Taylor
posted 15 Years Ago
ANSWER
Post Details
Share Post
Group: StrataFrame Developers
Posts: 6.6K,
Visits: 6.9K
I will try to get you a sample...I kind of think I have posted one out here before for Kenneth Langley. Might do a quick search to see if anything comes up. Also, we just called this method blob as some databases (i.e. Oracle) call it blob. But in essence you could do this for any field type. But in SQL Server, I would use a VarBinary(MAX) field to store the data. So you would just convert your file into a byte array and then save that into a VarBinary field.
I will try and allocate some time to write this sample...sorry for the delay. I am getting somewhat hammered this morning (and I am not referring to beer drinking contests!
)
Reply
Like
2
Edhy Rijo
E
Edhy Rijo
posted 15 Years Ago
ANSWER
Post Details
Share Post
E
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
Hi Trent,
Sorry to bother you on this one, but I searched the forums and could not find any sample code on how to do this. Please when you have some time see if you can post something quick to get the idea.
Thanks!
Edhy Rijo
Reply
Like
2
Trent Taylor
Trent Taylor
posted 15 Years Ago
ANSWER
Post Details
Share Post
Group: StrataFrame Developers
Posts: 6.6K,
Visits: 6.9K
I'm sorry Edhy...this slipped off of my radar. I will try and get you something right now.
I've been dealing with bureaucrats the last few days, so I was a bit preoccupied (not to mention frustrated!).
Reply
Like
2
Trent Taylor
Trent Taylor
posted 15 Years Ago
ANSWER
Post Details
Share Post
Group: StrataFrame Developers
Posts: 6.6K,
Visits: 6.9K
OK, here is an example I posted a while back. Let me know if this is not enough to work off of.
http://forum.strataframe.net/FindPost21474.aspx
Reply
Like
2
Edhy Rijo
E
Edhy Rijo
posted 15 Years Ago
ANSWER
Post Details
Share Post
E
Group: StrataFrame Users
Posts: 2.4K,
Visits: 23K
Thanks a lot, I believe that will be good enough, will try to implement this today and will post results.
About the bureaucrats, just don't pay too much attention to them
Edhy Rijo
Reply
Like
2
GO
Merge Selected
Merge into selected topic...
Merge into merge target...
Merge into a specific topic ID...
Open Merge
Threaded View
Threaded View
Saving big text files in a Varchar Max field?
Edhy Rijo
-
15 Years Ago
Hi Edhy, I also have the need for saving files (XML files)related to...
Teddy Jensen
-
15 Years Ago
[quote][b]Teddy Jensen (02/18/2010)[/b][hr]...I know your files are...
Edhy Rijo
-
15 Years Ago
Do you have tokeep the large text file in the table at all? Does the...
Russell Scott Brown
-
15 Years Ago
Edhy,
You are aware that you can update a single field as a...
Trent L. Taylor
-
15 Years Ago
FYI, the method name on the BO is PushBlobField or PushBlobFieldAsync....
Trent L. Taylor
-
15 Years Ago
[quote][b]Trent L. Taylor (02/18/2010)[/b][hr]Edhy,
You are...
Edhy Rijo
-
15 Years Ago
I will try to get you a sample...I kind of think I have posted one out...
Trent L. Taylor
-
15 Years Ago
Hi Trent,
Sorry to bother you on this one, but I searched...
Edhy Rijo
-
15 Years Ago
I'm sorry Edhy...this slipped off of my radar. I will try and get you...
Trent L. Taylor
-
15 Years Ago
OK, here is an example I posted a while back. Let me know if this is...
Trent L. Taylor
-
15 Years Ago
Thanks a lot, I believe that will be good enough, will try to...
Edhy Rijo
-
15 Years Ago
[quote][b]Russell Scott Brown (02/18/2010)[/b][hr]Do you have tokeep...
Edhy Rijo
-
15 Years Ago
Teddy's answer is what I was going to suggest. Try compressing the...
Dustin Taylor
-
15 Years Ago
Question: I am currently serializing PDF files to VARBINARY(max)...
Charles R Hankey
-
15 Years Ago
The technique your using Charles wouldn't do any compression. I.e. a...
Greg McGuffey
-
15 Years Ago
Well Charles, Greg,
Now that you mentioned compression, yes...
Edhy Rijo
-
15 Years Ago
That's what I'd try.
BTW, I took a look and my memory...
Greg McGuffey
-
15 Years Ago
Of course the question is also is the goal speed or size? Does the...
Charles R Hankey
-
15 Years Ago
[quote][b]Charles R Hankey (02/24/2010)[/b][hr]Of course the question...
Edhy Rijo
-
15 Years Ago
[quote] Thanks a lot, I believe that will be good enough, will try...
Teddy Jensen
-
15 Years Ago
[quote][b]Teddy Jensen (02/26/2010)[/b][hr]Edhy - You are normally...
Edhy Rijo
-
15 Years Ago
OK - thanks, was just curious on what route You choose. I will...
Teddy Jensen
-
15 Years Ago
[quote][b]Teddy Jensen (02/26/2010)[/b][hr]I will probably later try...
Edhy Rijo
-
15 Years Ago
This is all good stuff. One other thing I thought I would mention is...
Trent L. Taylor
-
15 Years Ago
OK - i tried to get this to work, but am apparently missing something,...
Teddy Jensen
-
15 Years Ago
Teddy, are you referring to the PullBlob or PushBlob here?
Trent L. Taylor
-
15 Years Ago
It was the PushBlob, but I have it working now. Ihad a null value...
Teddy Jensen
-
15 Years Ago
Great....glad you got it going! :) Yeah, it is really nice when you...
Trent L. Taylor
-
15 Years Ago
Post Reply
Like
2
Similar Topics
Post Quoted Reply
Reading This Topic
Login
Login
Remember Me
Reset Password
Resend Validation Email
Login
Explore
Messages
Mentions
Search