Article 3893 of comp.lang.perl:
Xref: feenix.metronet.com comp.lang.perl:3893
Path: feenix.metronet.com!news.ecn.bgu.edu!mp.cs.niu.edu!ux1.cso.uiuc.edu!howland.reston.ans.net!gatech!asuvax!ncar!vexcel!copper!aspen.craycos.com!scott
From:
[email protected] (Scott Bolte)
Newsgroups: comp.lang.perl
Subject: Re: ftpget script (provided)
Message-ID: <
[email protected]>
Date: 30 Jun 93 17:50:50 GMT
References: <
[email protected]>
Organization: Cray Computer Corporation
Lines: 832
In article <
[email protected]>
[email protected] (Mathias Koerber) writes:
...
>What I'd like is a script that just goes to a specified host, logs in,
>cd's to the directory and then ftps the named files. Would be nice if it
>sat atop ftp.pl.
>
>Does anyone already have such a beast?
I wrote just such a beast a while ago and have been using it
ever since. (In fact, it was used to pick up perl 4.036 just
two nights ago.) It does not use ftp.pl, all it requires is
ftp.
If you have the at command available it will even do the
transfers at night to ease the load on the net.
Here is a brief description of the scripts shar'ed together
below:
ftpget Obtains a single file from a remote host.
ftpmget Obtains a set of files from a remote host.
ftpls Run ls -l in a specific directory on a remote
host.
ftplist Given a set of hosts & directories make a
series of calls to ftpls. The results are
placed in files that reflect the listing
source.
Enjoy,
Scott
___________________________________________________________________________
Scott Bolte
[email protected] +1 719 540 4186
Cray Computer Corporation, 1110 Bayfield Drive, Colorado Springs, CO 80906
As anyone here will tell you: I speak for myself.
*** On the Internet no one can hear you scream ***
-------- rip, cut, sever, slice, dice or tear here --------------------------
#! /bin/sh
# This is a shell archive. Remove anything before this line, then unpack
# it by saving it into a file and typing "sh file". To overwrite existing
# files, type "sh file -c". You can also feed this as standard input via
# unshar, or by typing "sh <file", e.g.. If this archive is complete, you
# will see the following message at the end:
# "End of shell archive."
# Contents: README ftpget.pl ftplist.pl ftpls.pl ftpmget.pl
# Wrapped by scott@copper on Wed Jun 30 11:48:53 1993
PATH=/bin:/usr/bin:/usr/ucb ; export PATH
if test -f 'README' -a "${1}" != "-c" ; then
echo shar: Will not clobber existing file \"'README'\"
else
echo shar: Extracting \"'README'\" \(1371 characters\)
sed "s/^X//" >'README' <<'END_OF_FILE'
X Ftp Perl Scripts, Version 0.1
X
X Copyright (c) 1993, Scott Bolte
X
These scripts are free software. But if you want to send money anyway I
won't mind. The scripts are distributed in the hope that they will be
useful, but WITHOUT ANY WARRANTY.
X
You can take the scripts and use them yourself. You can even include
them in a product you sell as long as they do not add to the product
cost. But the copyright must be maintained.
X
The files in this directory were cobbled together in the last few
weeks. I have been using to automate ftp transfers. They also can be
asked to run at night in order to spread the demands on the net.
X
X ftpget Obtains a single file from a remote host.
X
X ftpmget Obtains a set of files from a remote host.
X
X ftpls Run ls -l in a specific directory on a remote
X host.
X
X ftplist Given a set of hosts & directories make a
X series of calls to ftpls. The results are
X placed in files that reflect the listing
X source.
X
X ftpvaporware A script, not yet written, that can look at the
X data provided by ftplist. It will provide
X notification that a item on an archive has
X changed. Or that a new release of a program is
X now available.
X
I would appreciate if any additions you make are sent to me. I will try
to incorporate them in any future releases.
X
X Scott Bolte (
[email protected])
X 1993.06.30
END_OF_FILE
if test 1371 -ne `wc -c <'README'`; then
echo shar: \"'README'\" unpacked with wrong size!
fi
# end of 'README'
fi
if test -f 'ftpget.pl' -a "${1}" != "-c" ; then
echo shar: Will not clobber existing file \"'ftpget.pl'\"
else
echo shar: Extracting \"'ftpget.pl'\" \(5666 characters\)
sed "s/^X//" >'ftpget.pl' <<'END_OF_FILE'
X#!/bin/perl
X
X#############################################################################
X#
X# Copyright 1993 Scott Bolte (
[email protected])
X#
X# Leave this copyright alone. But feel free to do with the script
X# as you please. Sending me enhancements would be appreciated.
X#
X# If you feel like pretending this is shareware, and want to
X# send some money my way, feel free. I promise not to object.
X#
X# Summary of ftpget:
X#
X# Obtain a file via ftp from a remote system. Anonymous ftp
X# is used. Can be asked to delay the actual ftp request.
X# On failure print out a call that can be used to try again.
X#
X# History:
X#
X# 1993.06.14 Initial implementation.
X#
X# 1993.06.15 Made remote and local file name processing safe
X# even when the file name contains white space.
X#
X# 1993.06.28 Added interface to use "at" internally.
X#
X# Examples:
X#
X# ftpget --at 23:30 prep.ai.mit.edu pub/gnu/perl-4.036.tar.gz
X#
X# ftpget --at 23:30 prep.ai.mit.edu pub/gnu/perl-4.036.tar.gz new-perl
X#
X# If you have an ftp hierarchy, as I do, the remote system can be
X# derived from the current path.
X#
X# cd ~/ftp/prep.ai.mit.edu
X# ftpget --at 23:30 - pub/gnu/perl-4.036.tar.gz
X#
X#############################################################################
X
X$zero = $0;
X$zero =~ s,.*/,,;
X
sub usage {
X print <<EOS;
Usage: $zero [options] system remote_file [local_file]
X -- Stop command line processing.
X -a or --at time At the given time, which should be in hh:mm format,
X run the $0 command.
X -d or --debug Do not run the actual ftp command, use cat instead.
X -v or --verbose Enable the verbose status message.
X -? or -h or --help Print this usage statement.
X
X If the system is "-" try to determine the remote system from
X the current path.
XEOS
X}
X
X#
X# Set the default values.
X#
X$verbose = 0;
X$debug = 0;
X$cmd = "ftp -n"; # change "ftp -n" to "cat -n" for testing.
X$time = time; # Get the time so we can recreate an "at"
X # command if need be.
X
X#
X# Process the command line.
X#
while ($ARGV[0] =~ /^-./) {
X $_ = shift;
X if (/^--$/) {
X last;
X }
X
X if (/^-a$/ || /^--at$/) {
X $delay_time = shift;
X if ( $delay_time !~ /^\d\d?:\d\d$/ ) {
X print(STDERR
X "Bad time specification \"$delay_time\".\n");
X &usage();
X exit(1);
X }
X next;
X }
X if (/^-d$/ || /^--debug$/) {
X $debug++;
X $cmd = "cat -n";
X next;
X }
X if (/^-v$/ || /^--verbose$/) {
X $verbose++;
X next;
X }
X
X if (/^-\?$/ || /^-h$/ || /^--help$/) {
X &usage();
X exit(0);
X }
X print "I don't recognize this switch: $_\n";
X &usage();
X exit(1);
X}
X
X#
X# Verify we have the right number of positional arguments.
X#
if ( $#ARGV < 1 || $#ARGV > 2 ) {
X &usage();
X exit(1);
X}
X$system = $ARGV[0];
X$remote_file = $ARGV[1];
X
X#
X# If the system spec was "-" try to figure out where we are. From that
X# we might be able to construct a default host.
X#
if ( $system eq "-" ) {
X $system = `/bin/pwd`;
X $system =~ s/\n//;
X $original = $system;
X $system =~ s,^.*/ftp/,,;
X $system =~ s,/.*$,,;
X die("Could not determine system given path \"$original\".\n")
X if $system eq "";
X print(STDERR "Derived system is \"$system\".\n") if $verbose;
X}
X
X#
X# Either take the optional third argument or construct it from the second.
X#
X
if ( "$remote_file" eq "" ) {
X print(STDERR "Must specify a non-null file name.\n");
X exit(1);
X}
if ( $#ARGV == 2 ) {
X $local_file = $ARGV[2];
X $explicit = 1;
X} else {
X $local_file = $remote_file;
X $local_file =~ s,/+$,,;
X $local_file =~ s,.*/,,;
X $explicit = 0;
X
X print(STDERR "Derived local path is \"$local_file\".\n") if $verbose;
X if ( "$local_file" eq "" ) {
X print(STDERR "Unable to construct a local filename.\n");
X exit(1);
X }
X}
X
X
X#
X# Make the path specifications safe even when they contain spaces.
X#
X$safe_remote = $remote_file;
X$safe_local = $local_file;
X$safe_remote =~ s/(.*)/"$1"/ if $safe_remote =~ /\s/;
X$safe_local =~ s/(.*)/"$1"/ if $safe_local =~ /\s/;
X
X#
X# If a delay was asked for run the command later.
X#
if ( $delay_time ne "" ) {
X $me = $0;
X $me .= " --verbose" if $verbose;
X $me .= " --debug" if $debug;
X $me .= " $system $safe_remote";
X $me .= " $safe_local" if $explicit;
X $at = "at $delay_time";
X print(STDERR "At $delay_time the following command will be run:\n");
X print(STDERR " $me\n");
X if ( $debug ) {
X print(STDERR "Skipping command in debug mode.\n");
X exit(0);
X }
X open(CMD, "|$at") || die("Could not run command ($at). $!\n");
X print(CMD $me);
X close(CMD);
X exit(0);
X}
X
X#
X# Build the batch ftp command.
X#
X$user = (getpwuid($<))[0]; # safe when run from "at".
X$localhost = `hostname`; chop($localhost);
if ( $localhost !~ /\./ ) {
X #
X # If the host name does not have '.' notation try to
X # get an alias. We then hope it is in domain name notation.
X #
X @fullhost = gethostbyname($localhost);
X $localhost = $fullhost[1] if $fullhost[1] ne "";
X}
X$template =
X" open $system
X user anonymous ${zero}4$user@$localhost
X bin
X get $safe_remote $safe_local
X bye
X";
X
X#
X# Run the batch ftp command.
X#
print(STDERR "Running command ($cmd).\n") if $verbose;
open(CMD, "|$cmd") || die("Could not start command ($cmd). $!\n");
print CMD $template;
close(CMD);
X
X
X# Note whether or not the local file was obtained.
X#
if ( -f $local_file ) {
X print("Obtained \"$local_file\" from $system.\n");
X} else {
X $pwd=`pwd`;
X chop($pwd);
X @time = localtime($time);
X $next = sprintf("%2d:%02d", $time[2], $time[1]);
X
X $sep = "\t\\\n\t ";
X $cmd = "$0";
X $cmd .= "$sep--at $next";
X $cmd .= "$sep$system";
X $cmd .= "$sep$safe_remote";
X $cmd .= "$sep$safe_local" if $explicit;
X
X print("
Unable to obtained \"$local_file\" from $system.
To try again the following command might be used:
X
cd $pwd;
X$cmd
X");
X}
END_OF_FILE
if test 5666 -ne `wc -c <'ftpget.pl'`; then
echo shar: \"'ftpget.pl'\" unpacked with wrong size!
fi
chmod +x 'ftpget.pl'
# end of 'ftpget.pl'
fi
if test -f 'ftplist.pl' -a "${1}" != "-c" ; then
echo shar: Will not clobber existing file \"'ftplist.pl'\"
else
echo shar: Extracting \"'ftplist.pl'\" \(1543 characters\)
sed "s/^X//" >'ftplist.pl' <<'END_OF_FILE'
X#!/bin/perl
X
X#############################################################################
X#
X# Copyright 1993 Scott Bolte (
[email protected])
X#
X# Leave this copyright alone. But feel free to do with the script
X# as you please. Sending me enhancements would be appreciated.
X#
X# If you feel like pretending this is shareware, and want to
X# send some money my way, feel free. I promise not to object.
X#
X# Summary of ftplist:
X#
X# Run ftpls on a bunch of systems. The results are put in files
X# whose names map to the system/directory pair. Older copies are
X# renamed before the new edition is obtained.
X#
X# The expectation is that additional scripts will be run after
X# this one. They will compare the old and new listings to note
X# changes.
X#
X# History:
X#
X# 1993.06.28 Initial implementation.
X#
X# Examples:
X#
X# ftplist
X#
X#############################################################################
X
X%set = (
X "agate.berkeley.edu", "pub/386BSD/386bsd-0.1/unofficial",
X "bsd.coe.montana.edu", "pub/patch-kit",
X "hrd769.brooks.af.mil", "pub/FAQ",
X "prep.ai.mit.edu", "pub/gnu",
X );
X
foreach $system (sort(keys(%set))) {
X $file = "$system:$set{$system}";
X $file =~ s,/,_,g;
X $old = "$file.OLD";
X if ( -f $file ) {
X $error = "Could not rename \"$file\" to \"$old\". $!\n";
X $error .= "New listing of $set{$system} will not be obtained.\n";
X rename($file, $old) || (warn($error), next);
X }
X $cmd = "ftpls $system $set{$system} > $file";
X system($cmd) && die("Could not run command ($cmd). $!\n");
X}
END_OF_FILE
if test 1543 -ne `wc -c <'ftplist.pl'`; then
echo shar: \"'ftplist.pl'\" unpacked with wrong size!
fi
# end of 'ftplist.pl'
fi
if test -f 'ftpls.pl' -a "${1}" != "-c" ; then
echo shar: Will not clobber existing file \"'ftpls.pl'\"
else
echo shar: Extracting \"'ftpls.pl'\" \(1853 characters\)
sed "s/^X//" >'ftpls.pl' <<'END_OF_FILE'
X#!/bin/perl
X
X#############################################################################
X#
X# Copyright 1993 Scott Bolte (
[email protected])
X#
X# Leave this copyright alone. But feel free to do with the script
X# as you please. Sending me enhancements would be appreciated.
X#
X# If you feel like pretending this is shareware, and want to
X# send some money my way, feel free. I promise not to object.
X#
X# Summary of ftpls:
X#
X# Do a ls in a specific directory on a remote ftp system.
X# Anonymous ftp is used.
X#
X# History:
X#
X# 1993.06.28 Initial implementation.
X#
X# Examples:
X#
X# ftpls remote_system remote_dir
X#
X#############################################################################
X
X$cmd = "ftp -n"; # change "ftp -n" to "cat -n" for testing.
X$zero = $0;
X$zero =~ s,.*/,,;
X
X#############################################################################
X#
X# Verify the arguments
X#
if ( $#ARGV != 1 ) {
X print(STDERR "Usage: $zero system remote_dir\n");
X exit(1);
X}
X
X$system = $ARGV[0];
X$remote_dir = $ARGV[1];
X
X#############################################################################
X#
X# Build the batch ftp command.
X#
X$user = (getpwuid($<))[0]; # safe when run from "at".
X$localhost = `hostname`; chop($localhost);
if ( $localhost !~ /\./ ) {
X #
X # If the host name does not have '.' notation try to
X # get an alias. We then hope it is in domain name notation.
X #
X @fullhost = gethostbyname($localhost);
X $localhost = $fullhost[1] if $fullhost[1] ne "";
X}
X$template =
X" open $system
X user anonymous ${zero}4$user@$localhost
X bin
X cd $remote_dir
X ls -l
X bye
X";
X
X#############################################################################
X#
X# Run the batch ftp command.
X#
open(CMD, "|$cmd") || die("Could not start command ($cmd). $!\n");
print CMD $template;
close(CMD);
X
exit(0);
END_OF_FILE
if test 1853 -ne `wc -c <'ftpls.pl'`; then
echo shar: \"'ftpls.pl'\" unpacked with wrong size!
fi
chmod +x 'ftpls.pl'
# end of 'ftpls.pl'
fi
if test -f 'ftpmget.pl' -a "${1}" != "-c" ; then
echo shar: Will not clobber existing file \"'ftpmget.pl'\"
else
echo shar: Extracting \"'ftpmget.pl'\" \(7128 characters\)
sed "s/^X//" >'ftpmget.pl' <<'END_OF_FILE'
X#!/bin/perl
X
X#############################################################################
X#
X# Copyright 1993 Scott Bolte (
[email protected])
X#
X# Leave this copyright alone. But feel free to do with the script
X# as you please. Sending me enhancements would be appreciated.
X#
X# If you feel like pretending this is shareware, and want to
X# send some money my way, feel free. I promise not to object.
X#
X# Summary of ftpmget:
X#
X# Obtain a set of files via ftp from a remote system. Anonymous
X# ftp is used. Can be asked to delay the command until a later
X# time. Such a request will result in "at" being used.
X#
X# History:
X#
X# 1993.06.14 Initial implementation.
X#
X# 1993.06.15 Changed so that if a file contains white space
X# it is obtained with get instead of mget.
X#
X# 1993.06.30 Added --at option to allow delayed operation.
X#
X# Examples:
X#
X# ftpmget hrd769.brooks.af.mil pub/FAQ FAQ_07 FAQ_09
X#
X# cd ~/ftp/hrd769.brooks.af.mil
X# ftpmget - pub/FAQ FAQ_07 FAQ_09
X#
X# ftpmget hrd769.brooks.af.mil pub/FAQ - << EndOfList
X# FAQ_07
X# FAQ_09
X# EndOfList
X#
X#############################################################################
X
X$zero = $0;
X$zero =~ s,.*/,,;
X
sub usage {
X
X print <<EOS;
Usage: $zero [options] system remote_dir file1 [... fileN]
X or
X $zero [options] system remote_dir -
X
X -- Stop command line processing.
X -a or --at time At the given time, which should be in hh:mm format,
X run the $0 command.
X -d or --debug Do not run the actual ftp command, use cat instead.
X -v or --verbose Enable the verbose status message.
X -? or -h or --help Print this usage statement.
X
X If the system is "-" try to determine the remote system from
X the current path.
X
X If a "-" is given instead of a list of files the list is read
X from standard input.
XEOS
X}
X
X##############################################################################
X#
X# Set the default values.
X#
X
X$verbose = 0;
X$debug = 0;
X$cmd = "ftp -n"; # change "ftp -n" to "cat -n" for testing.
X$time = time; # Get the time so we can recreate an "at"
X # command if need be.
X$pwd =`/bin/pwd`; chop($pwd);
X
X##############################################################################
X#
X# Process the command line.
X#
X
while ($ARGV[0] =~ /^-./) {
X $_ = shift;
X if (/^--$/) {
X last;
X }
X
X if (/^-a$/ || /^--at$/) {
X $delay_time = shift;
X if ( $delay_time !~ /^\d\d?:\d\d$/ ) {
X print(STDERR
X "Bad time specification \"$delay_time\".\n");
X &usage();
X exit(1);
X }
X next;
X }
X if (/^-d$/ || /^--debug$/) {
X $debug++;
X $cmd = "cat -n";
X next;
X }
X if (/^-v$/ || /^--verbose$/) {
X $verbose++;
X next;
X }
X
X if (/^-\?$/ || /^-h$/ || /^--help$/) {
X &usage();
X exit(0);
X }
X print "I don't recognize this switch: $_\n";
X &usage();
X exit(1);
X}
X
X##############################################################################
X#
X# Verify the positional arguments
X#
if ( $#ARGV < 2 ) {
X &usage();
X exit(1);
X}
X
X$system = $ARGV[0];
X$remote_dir = $ARGV[1];
X
X##############################################################################
X#
X# If the system spec was "-" try to figure out where we are. From that
X# we might be able to construct a default host.
X#
if ( $system eq "-" ) {
X $system = `/bin/pwd`;
X $system =~ s/\n//;
X $original = $system;
X $system =~ s,^.*/ftp/,,;
X $system =~ s,/.*$,,;
X die("Could not determine system given path \"$original\".\n")
X if $system eq "";
X print(STDERR "Derived system is \"$system\".\n") if $verbose;
X}
X
X##############################################################################
X#
X# Get the list of requested files. Either take the files from
X# the command line or from stdin.
X#
if ( $ARGV[2] ne "-" ) {
X for( $argc = 2; $argc <= $#ARGV; $argc++) {
X $file = $ARGV[$argc];
X push(@files, $file);
X }
X} else {
X while(<STDIN>) {
X chop;
X $file = $_;
X $file =~ s/^\s+//; # leading white space
X $file =~ s/\s+$//; # trailing white space
X $file =~ s/^"(.*)"$/$1/; # Remove enclosing quotes
X push(@files, $file);
X }
X}
X
X#
X# Make spaces safe for all mankind. N. Armstrong.
X#
foreach $file (@files) {
X next if $file eq ""; # skip empty names
X $file =~ s/(.*)/"$1"/ if $file =~ /\s/; # add quotes if need be.
X push(@tmp, $file);
X}
X@files = @tmp;
undef @tmp;
X
X##############################################################################
X#
X# If a delay was asked for run the command later.
X#
X
if ( $delay_time ne "" ) {
X $me = $0;
X $me .= " --verbose" if $verbose;
X $me .= " --debug" if $debug;
X $me .= " $system $remote_dir - << End_Of_List\n";
X foreach $file (@files) {
X $me .= "\t$file\n";
X }
X $me .= "End_Of_List\n";
X
X $at = "at $delay_time";
X
X print(STDERR "At $delay_time the following command will be run:\n");
X print(STDERR " $me\n");
X if ( $debug ) {
X print(STDERR "Skipping command in debug mode.\n");
X exit(0);
X }
X open(CMD, "|$at") || die("Could not run command ($at). $!\n");
X print(CMD $me);
X close(CMD);
X exit(0);
X}
X
X##############################################################################
X#
X# Build the batch ftp command.
X#
X
X$user = (getpwuid($<))[0]; # safe when run from "at".
X$localhost = `hostname`;
X chop($localhost);
if ( $localhost !~ /\./ ) {
X #
X # If the host name does not have '.' notation try to
X # get an alias. We then hope it is in domain name notation.
X #
X @fullhost = gethostbyname($localhost);
X $localhost = $fullhost[1] if $fullhost[1] ne "";
X}
X$template =
X" open $system
X user anonymous ${zero}4$user@$localhost
X bin
X prompt
X cd $remote_dir
X";
foreach $file (@files) {
X if ( $file =~ /\s/ ) {
X push(@space_files, $file);
X next;
X }
X if ( length($line) + length($file) + 1 > 75) {
X $template .= sprintf("$line\n");
X $line = "";
X }
X $line = " mget" if $line eq "";
X $line .= " $file";
X}
X$template .= "$line\n";
foreach $file (@space_files) {
X $template .= " get $file\n";
X}
X$template .= "$bye\n";
X
X##############################################################################
X#
X# Run the batch ftp command.
X#
X
open(CMD, "|$cmd") || die("Could not start command ($cmd). $!\n");
print CMD $template;
close(CMD);
X
X##############################################################################
X#
X# Report whether or not the files were obtained.
X#
X
print("\nReport for file transfers from $system.\n");
print(" Remote directory \"$remote_dir\".\n");
print(" Local directory \"$pwd\".\n");
foreach $file (@files) {
X if ( -f $file ) {
X print(" Obtained \"$file\".\n");
X next;
X }
X
X if ( $file =~ /^".*"$/ ) {
X $file =~ s/^"(.*)"$/$1/;
X if ( -f $file ) {
X print(" Obtained \"$file\".\n");
X next;
X }
X }
X push(@again, $file);
X}
X
X##############################################################################
X#
X# If there was a problem getting all the files print out a command
X# that can try again later.
X#
X
if ( $#again >= $[ ) {
X @time = localtime($time);
X $next = sprintf("%2d:%02d", $time[2], $time[1]);
X
X print <<EOS;
X
Unable to obtained some files from $system.
To try again the following command might be used:
X
cd $pwd;
X$0 --at $next $system $remote_dir - << End_Of_List
XEOS
X
X foreach $file (@again) {
X print(" $file\n");
X }
X print("End_Of_List\n");
X}
END_OF_FILE
if test 7128 -ne `wc -c <'ftpmget.pl'`; then
echo shar: \"'ftpmget.pl'\" unpacked with wrong size!
fi
chmod +x 'ftpmget.pl'
# end of 'ftpmget.pl'
fi
echo shar: End of shell archive.
exit 0
--
___________________________________________________________________________
Scott Bolte
[email protected] +1 719 540 4186
Cray Computer Corporation, 1110 Bayfield Drive, Colorado Springs, CO 80906
As anyone here will tell you: I speak for myself.
*** On the Internet no one can hear you scream ***