Of course the question is also is the goal speed or size? Does the trade-off of serializing and deserializing the whitespace - resulting in a large piece of data - payoff in faster in and out time vs the time it takes to compress/uncompress.



I don't know the answer, obviously, but it seems to be part of the question.

That's what I'd try.



BTW, I took a look and my memory failed me. There is some compression stuff in the StrataFrame.IO.Compression namespace, but it is all related to creating packages (very handy for ziping multiple files into one file) and not what we are talking about (compressing/decompressing a stream).



It turns out that it is the .NET library in the IO.Compression namespace. Here are some links related to this:



MS info on this namespace. We're interested in the DeflateStream and GZipStream classes:

http://msdn.microsoft.com/en-us/library/system.io.compression.aspx



This one explains a bit and then provides a utility to handle zip files:

http://blogs.msdn.com/dotnetinterop/archive/2006/04/05/.NET-System.IO.Compression-and-zip-files.aspx

Well Charles, Greg,



Now that you mentioned compression, yes that is something I would like, since my files are just basically text files I am sure any kind of compression will help.



Since I have not tried any of the techniques, I believe I would have to do the following:



1.- Read the text file to stream

2.- Compress that stream

3.- Save it to the Varbinary(max) field using Trent's technique.



Am I missing something here?

Edhy Rijo

The technique your using Charles wouldn't do any compression. I.e. a space is ASCII 32, data just like anything else. Now if the file were using some sort of compression algorithm, then you'd get that. I.e. JPG, PNG etc have compression built into them. So, if the PDF file format includes some compression, then you'd get that compression. I think it's a pretty simple call to use the zip tools in the framework to compress the resulting byte stream. I'd check out the tools area, if memory is correct. I check into this later today, as I'm a curious guy.

Question: I am currently serializing PDF files to VARBINARY(max) fields. Of course they are not as large as what Edhy is talking about. In the context of this question, do I understand correctly that just doing this doesn't compress the whitespace out of the file and that is why the more elaborate technique is necessary, or am I just not understanding the issue at all ?



For the PDFs I am just using

Thanks a lot, I believe that will be good enough, will try to implement this today and will post results.



About the bureaucrats, just don't pay too much attention to them

Edhy Rijo

OK, here is an example I posted a while back. Let me know if this is not enough to work off of.



http://forum.strataframe.net/FindPost21474.aspx

I'm sorry Edhy...this slipped off of my radar. I will try and get you something right now.



I've been dealing with bureaucrats the last few days, so I was a bit preoccupied (not to mention frustrated!).

Hi Trent,



Sorry to bother you on this one, but I searched the forums and could not find any sample code on how to do this. Please when you have some time see if you can post something quick to get the idea.



Thanks!

Edhy Rijo

I will try to get you a sample...I kind of think I have posted one out here before for Kenneth Langley. Might do a quick search to see if anything comes up. Also, we just called this method blob as some databases (i.e. Oracle) call it blob. But in essence you could do this for any field type. But in SQL Server, I would use a VarBinary(MAX) field to store the data. So you would just convert your file into a byte array and then save that into a VarBinary field.



I will try and allocate some time to write this sample...sorry for the delay. I am getting somewhat hammered this morning (and I am not referring to beer drinking contests! )