README for Perl module String::Tokenizer
This module provides a simple string tokenizer which takes a string
and splits it on whitespace. It also optionally takes a string of
characters to use as delimiters, and returns them with the token set
as well. This allows for splitting the string in many different ways.
You can read a nicely formatted version of the documentation for
this module online:
https://metacpan.org/pod/String::Tokenizer
You should be able to install this using your usual method for installing
modules from CPAN. If you don't have one, have a look at:
http://www.cpan.org/modules/INSTALL.html
This module was written by Stevan Little <
[email protected]>
Copyright (c) 2004-2016 Infinity Interactive, Inc.