Subj : file size issues?
To : Roy J. Tellason
From : Mike Tripp
Date : Sun Aug 01 2004 12:08 pm
Hello Roy!
01 Aug 04 04:07, Roy J. Tellason wrote to Mike Tripp:
RT> I'm seeing the same numbers in both cases there because I have no
RT> settings in the area.
Lack of settings alone won't keep the numbers static. If you manually kill one
message and SQPACK again, Old should be larger than New. If you manually edit
one existing message to remove a few lines from the message, Old should be
larger than New.
RT> Right. But to get to that point I had to *move* the remaining
RT> messages I wanted to keep from one area to another.
Gotcha...and obviously the stats you're posting are for the new version of the
area and not the original hosed version. It is the hosed version that
should've been shrinking as messages were moved out of it. The new version was
only growing to swallow the new messages being appended, so there's no reason
for them to be occupying more than the actual number of bytes required to do
so.
RT> All with today's date, and a timestamp within the past hour when I
RT> was reading it last. What's that sqi file used for anyhow?
Those that've been into the source up to their elbows can probably find plenty
of holes with my analogy, but if you understand the basics of how DOS manages
files and space allocated on a FAT hard drive: the .SQI is the FAT table for
the files (messages) within Squish's hard drive (.SQD).
The .SQI basically has the pointers to where the header for each message starts
and how many blocks it occupies. If you delete a message, Squish just updates
the .SQI file that references the deleted data to remove the pointer to its
header and updates it's usage map to show that space as available for writing
new data again. The size of the .SQD is not necessarily representative of the
actual number of bytes required by the currently retained messages. Squish
will expand the .SQD if more space is needed to accomodate a new retention
peak, but it will not shrink it just because fewer are required today than
yesterday.
Just as bad things can happen to a FAT if operations fail or are interrupted by
a system lockup or power failure, the same is true for Squish and .SQI/.SQD
manipulations. SQFIX does for the Squishbase what CHKDSK and/or Disk Doctor
does for the integrity of the FAT. It looks to make sure that a message header
actually starts where each .SQI entry says it does and looks for space that the
.SQI says should be in use by a retained message but actually isn't (akin to
lost clusters and cross-linked files) and attempts to either fix or remove .SQI
entries that don't seem to match reality found in the .SQD.
SQPACK is analagous to DEFRAG. If you have specified parameters, it first
marks the messages that fail those parameters as deleted and then "compresses"
the .SQD data to the beginning of the .SQD file and shrinks the .SQD file down
to the actual size required to store the actual bytes associated with those
messages that are still retained, rather than the size being some former
worst-case maximum that was required to store messages that met the SQSET
parameters at some point in the past since you last SQPACKed.
So if you use no purge parameters, never manually delete/edit a message in an
area, and never have any problem that causes the .SQI to lose synch with the
actual contents of the .SQD, it will appear that SQPACK never does
anything...just as running DEFRAG wouldn't accomplish anything for your hard
drive if you only appended new files one at a time and never deleted or
modified an existing file since your last DEFRAG. If, however, you make any of
those changes or SQFIX frees a pointer to an existing message because of a data
integrity issue, it is possible/probable that there will be some delta between
the old/new bytes when you SQPACK next, even if you haven't assigned specific
purge criteria to the area.
