Proposed Guidelines for the FileBone
Erik VanRiper
1:107/230
Status of this document:
This FSC suggests a proposed protocol for the FidoNet(r) community,
and requests discussion and suggestions for improvements.
Distribution of this document is unlimited.
Fido and FidoNet are registered marks of Tom Jennings and Fido
Software.
1. Purpose.
The purpose of this document is to set down basic guidelines for the
handling of File Distribution Networks on a "File Distribution
Backbone".
2. Definition of Terms.
a. FDN. FDN is a File Distribution Network, made up of at least one
file area dedicated to moving files through Fidonet compatible
mailers for other nodes to utilize. An example of this is the
Software Distribution Network or SDS as it is more commonly known.
b. FTN. FTN is a term coined to signify that another Network besides
FidoNet has the same technology as FidoNet, and can transfer files
and mail via FTSC-001 compatible mailers. An example of this is
SigNet.
c. TICK. TICK (and HATCH) is (C) Copyright Barry Geller - 1988, 1989,
1990, 1991, 1992. TICK is the current popular way to move files in
FTN's participating in File Distribution.
d. FILEBONE. FILEBONE is the "File Distribution Backbone". The term
FILEBONE is used in place of BACKBONE because the BackBone is used
for transfering mail, not files. There is a seperate document for
procedures of FidoNet BackBone systems.
3. Reasons for this Document.
a. Spending the last five months compiling information on all the
FDN's and how the files are moving in each, I noticed that a lot of
time, money, and hassle can be avoided by creating a well defined
mechinism by which all FDN's can participate. In the past, (and
currently), there has been more concern by the heads of individual
FDN's as to WHO is picking up their FDN, and not enough concern in
how FAST those nodes are getting newly hatched files. This
document will attempt to address both issues.
b. Personally, I feel that there is not enough credit given to those
major file hubs all over the world who pay an arm and a leg to pick
up different FDN's from several different locations to support 50
or more FTN systems they have polling for support. These HUMANS
have always worked hard, and received almost no credit. I would
just like to take time out in this document to thank each and every
one of them for the wonderful support they have contributed to
several different FTN's.
4. The Outline.
a. The FILEBONE will be created in several stages, taking up to a year
to become fully operational. This will require several support
programs to be written, tested, and documented, as well as
re-arranging current FDN links to test speed and reliability of
actually moving the files.
b. The FILEBONE will consist of no less than 15 systems in Zone 1 (I
cannot speak for other zones). Each system will be required to
make at least one call a night to drop off and pick up ALL the
available FDN's that are participating in the FILEBONE. These
FILEBONE sites will also be primary hubs (acting as "stars") for
other FTN compatible systems interested in obtaining parts (or all)
of the files transferred.
c. Each FILEBONE site will use several different programs written to
aid in locating delays, problems, and undesireable conditions while
processing the files. They will also be required to submit "File
Distribution Reports" each week to the FILEBONE database, which
will maintain and analyze the information, detecting possible
problem areas.
d. Each FILEBONE site will be required to submit to the wishes of each
FDN concerning hatching policies, linking policies, cutting of
links to problem nodes, and general reporting of usage. FILEBONE
sites are not "pawns" of individual FDN's, but "tools" for each FDN
to use to get their files (and support conferences) from one end of
the FILEBONE to the other.
e. Each FDN will be required to submit a statment of agreement to this
document to the FILEBONE systems, as the FILEBONE systems are the
ones paying to move their files.
f. The FILEBONE will consist of a two-tierd system. The largest being
the actualy FILEBONE, where all the "released" files will be
transported. The second smaller level will be a "back area" for
each FDN that requires one. The concept is this: If a system
hatches a file in area GENERAL, and Joe Smith, the moderator of the
GENERAL area has not authorized that system to hatch into that
area, the first FILEBONE site to get this file will move that file
to the GENERAL "back area" for review by Joe Smith. Once Joe Smith
decides on suitability, he will then send a message back to that
FILEBONE site (and all other in-between) saying it is OK to let the
file pass, delete the file, or alter the description of that file
before letting it pass. The FILEBONE will move the file to that
FDN's moderator on the "back area", so all FILEBONE sites that have
already seen the file can simply "move" that file back into
distribution, so that those FILEBONE sites already having the file
will not need to re-transfer it. This system ensures that there is
only a small delay in time for checking the validity of that file.
A basic diagram follows:
System A hatches file "FILENAME.ZIP" into the GENERAL file area.
1:0/0 detects that system is not on the list authorized to hatch
files into the GENERAL file area, so sends the file to 1:0/1 in the
GENERAL backarea, enroute to 0:5/5. When 0:5/5 gets FILENAME.ZIP,
1:0/0, 1:0/1, 1:0/2, and 1:0/3 will have already seen the file, and
1:0/4 has not. Joe Smith (The moderator of the GENERAL file area),
at 0:5/5 will test FILENAME.ZIP to see if it is acceptable for
distribution. If it is, Joe Smith will send a netmail message back
to a program running on 1:0/3, letting the FILEBONE know it is OK
to distribute the file. 1:0/3 will then move the file to the
GENERAL area, and continue to send it on. 1:0/3 will also generate
a message back to 1:0/2 letting them know the status of the file,
and so on, until 1:0/0 has finally moved the file back into the
GENERAL area. Another options to Joe Smith are to have the file
deleted (Usually because it has been duplicated), or to have the
file fowarded to another FDN moderator, where the file would be
more suitable. With this method of checking files, FDN's can allow
the FILEBONE to let anyone hatch files into their FDN without
having to worry about duplicates or programs that are not suitable
for that FDN.
g. FILEBONE sites will also be required to keep an online database
available to any nodes requiring information about individual FDN
areas. This information will include:
1. Average traffic per week and month.
2. Average time to obtain file submitted.
3. A listing of nodes carrying that FDN area.
4. Guidelines, applications, and policies associated with that
FDN.
This will be an automated process, carried out much like a node
sending an AREAFIX or RAID request. The FILEBONE site being
queried for the information is only resposible for 2 things:
1. Ensuring the database is operational.
2. Placing the requested information on hold for pickup.
Anything else that FILEBONE site does with the request for
information is at their disgression, such as sending back the
information on their dime.
h. All FILEBONE sites will be required to drop the individual "User
Flags" in the nodelist that corespond to individual FDN's (in
regions that allow Uxxx flags in the nodelist). They will instead
use the "UFDN" flag. This will help (albeit a small amount) cut
down on the flag usage in the nodelist, since all the FILEBONE
sites will be moving most of the available file areas.
5. The FILEBONE has started, with 21 systems in Zone 1, one system in
Zone 2, and several "OtherNets" getting involved by the day. Several
of the programs outlined in this document have already been written,
and are in use, being tested. There are still a few more programs to
be written, but things are running smoothly as of the date on this
document. 99% of the known FDN's in FidoNet are linked into the
FILEBONE in one form or another, and the future looks very promising.
This document is by no means complete. There are several other
aspects to FDN's and FILEBONE that are not discussed here. This
document is put forth for comments, additions, deletions, and all
general changes. This is only how I (the author of this document)
envision the FILEBONE operating, and this document may be flawed in
one or several areas.
