SYNOPSIS

       use Web::Query;

       wq('http://www.w3.org/TR/html401/')
           ->find('div.head dt')
           ->each(sub {
               my $i = shift;
               printf("%d %s\n", $i+1, $_->text);
           });

DESCRIPTION

   Web::Query is a yet another scraping framework, have a jQuery like
   interface.

   Yes, I know Ingy's pQuery. But it's just a alpha quality. It doesn't
   works. Web::Query built at top of the CPAN modules,
   HTML::TreeBuilder::XPath, LWP::UserAgent, and HTML::Selector::XPath.

   So, this module uses HTML::Selector::XPath and only supports the CSS 3
   selector supported by that module. Web::Query doesn't support jQuery's
   extended queries(yet?). If a selector is passed as a scalar ref, it'll
   be taken as a straight XPath expression.

       $wq( '<div><p>hello</p><p>there</p></div>' )->find( 'p' );       # css selector
       $wq( '<div><p>hello</p><p>there</p></div>' )->find( \'/div/p' ); # xpath selector

   THIS LIBRARY IS UNDER DEVELOPMENT. ANY API MAY CHANGE WITHOUT NOTICE.

FUNCTIONS

   wq($stuff)

     This is a shortcut for Web::Query->new($stuff). This function is
     exported by default.

METHODS

CONSTRUCTORS

   my $q = Web::Query->new($stuff, \%options )

     Create new instance of Web::Query. You can make the instance from
     URL(http, https, file scheme), HTML in string, URL in string, URI
     object, undef, and either one HTML::Element object or an array ref of
     them.

         # all valid creators
         $q = Web::Query->new( 'http://techblog.babyl.ca' );
         $q = Web::Query->new( '<p>foo</p>' );
         $q = Web::Query->new( undef );

     This method throw the exception on unknown $stuff.

     This method returns undefined value on non-successful response with
     URL.

     Currently, the only two valid options are indent, which will be used
     as the indentation string if the object is printed, and
     no_space_compacting, which will prevent the compaction of whitespace
     characters in text blocks.

   my $q = Web::Query->new_from_element($element: HTML::Element)

     Create new instance of Web::Query from instance of HTML::Element.

   my $q = Web::Query->new_from_html($html: Str)

     Create new instance of Web::Query from HTML.

   my $q = Web::Query->new_from_url($url: Str)

     Create new instance of Web::Query from URL.

     If the response is not success(It means /^20[0-9]$/), this method
     returns undefined value.

     You can get a last result of response, use the $Web::Query::RESPONSE.

     Here is a best practical code:

         my $url = 'http://example.com/';
         my $q = Web::Query->new_from_url($url)
             or die "Cannot get a resource from $url: " . Web::Query->last_response()->status_line;

   my $q = Web::Query->new_from_file($file_name: Str)

     Create new instance of Web::Query from file name.

TRAVERSING

 add

   Returns a new object augmented with the new element(s).

   add($html)

     An HTML fragment to add to the set of matched elements.

   add(@elements)

     One or more @elements to add to the set of matched elements.

     @elements that already are part of the set are not added a second
     time.

         my $group = $wq->find('#foo');         # collection has 1 element
         $group = $group->add( '#bar', $wq );   # 2 elements
         $group->add( '#foo', $wq );            # still 2 elements

   add($wq)

     An existing Web::Query object to add to the set of matched elements.

   add($selector, $context)

     $selector is a string representing a selector expression to find
     additional elements to add to the set of matched elements.

     $context is the point in the document at which the selector should
     begin matching

 contents

   Get the immediate children of each element in the set of matched
   elements, including text and comment nodes.

 each

   Visit each nodes. $i is a counter value, 0 origin. $elem is iteration
   item. $_ is localized by $elem.

       $q->each(sub { my ($i, $elem) = @_; ... })

 end

   Back to the before context like jQuery.

 filter

   Reduce the elements to those that pass the function's test.

       $q->filter(sub { my ($i, $elem) = @_; ... })

 find

   Get the descendants of each element in the current set of matched
   elements, filtered by a selector.

       my $q2 = $q->find($selector); # $selector is a CSS3 selector.


   NOTE If you want to match the element itself, use "filter".

   INCOMPATIBLE CHANGE From v0.14 to v0.19 (inclusive) find() also matched
   the element itself, which is not jQuery compatible. You can achieve
   that result using filter(), add() and find():

       my $wq = wq('<div class="foo"><p class="foo">bar</p></div>'); # needed because we don't have a global document like jQuery does
       print $wq->filter('.foo')->add($wq->find('.foo'))->as_html; # <div class="foo"><p class="foo">bar</p></div><p class="foo">bar</p>

 first

   Return the first matching element.

   This method constructs a new Web::Query object from the first matching
   element.

 last

   Return the last matching element.

   This method constructs a new Web::Query object from the last matching
   element.

 match($selector)

   Returns a boolean indicating if the elements match the $selector.

   In scalar context returns only the boolean for the first element.

   For the reverse of not(), see filter().

 not($selector)

   Returns all the elements not matching the $selector.

       # $do_for_love will be every thing, except #that
       my $do_for_love = $wq->find('thing')->not('#that');

 and_back

   Add the previous set of elements to the current one.

       # get the h1 plus everything until the next h1
       $wq->find('h1')->next_until('h1')->and_back;

 map

   Creates a new array with the results of calling a provided function on
   every element.

       $q->map(sub { my ($i, $elem) = @_; ... })

 parent

   Get the parent of each element in the current set of matched elements.

 prev

   Get the previous node of each element in the current set of matched
   elements.

       my $prev = $q->prev;

 next

   Get the next node of each element in the current set of matched
   elements.

      my $next = $q->next;

 next_until( $selector )

   Get all subsequent siblings, up to (but not including) the next node
   matched $selector.

MANIPULATION

 add_class

   Adds the specified class(es) to each of the set of matched elements.

       # add class 'foo' to <p> elements
       wq('<div><p>foo</p><p>bar</p></div>')->find('p')->add_class('foo');

 toggle_class( @classes )

   Toggles the given class or classes on each of the element. I.e., if the
   element had the class, it'll be removed, and if it hadn't, it'll be
   added.

   Classes are toggled once, no matter how many times they appear in the
   argument list.

       $q->toggle_class( 'foo', 'foo', 'bar' );

       # equivalent to

       $q->toggle_class('foo')->toggle_class('bar');

       # and not

       $q->toggle_class('foo')->toggle_class('foo')->toggle_class('bar');

 after

   Insert content, specified by the parameter, after each element in the
   set of matched elements.

       wq('<div><p>foo</p></div>')->find('p')
                                  ->after('<b>bar</b>')
                                  ->end
                                  ->as_html; # <div><p>foo</p><b>bar</b></div>


   The content can be anything accepted by "new".

 append

   Insert content, specified by the parameter, to the end of each element
   in the set of matched elements.

       wq('<div></div>')->append('<p>foo</p>')->as_html; # <div><p>foo</p></div>


   The content can be anything accepted by "new".

 as_html

   Returns the string representations of either the first or all elements,
   depending if called in list or scalar context.

   If given an argument join, the string representations of the elements
   will be concatenated with the given string.

       wq( '<div><p>foo</p><p>bar</p></div>' )
           ->find('p')
           ->as_html( join => '!' );
       # <p>foo</p>!<p>bar</p>

  attr

   Get/set attribute values.

   In getter mode, it'll return either the values of the attribute for all
   elements of the set, or only the first one depending of the calling
   context.

       my @values = $q->attr('style');      # style of all elements
       my $first_value = $q->attr('style'); # style of first element

   In setter mode, it'll set attributes value for all elements, and return
   back the original object for easy chaining.

       $q->attr( 'alt' => 'a picture' )->find( ... );

       # can pass more than 1 element too
       $q->attr( alt => 'a picture', src => 'file:///...' );

   The value passed for an attribute can be a code ref. In that case, the
   code will be called with $_ set to the current attribute value. If the
   code modifies $_, the attribute will be updated with the new value.

       $q->attr( alt => sub { $_ ||= 'A picture' } );

  id

   Get/set the elements's id attribute.

   In getter mode, it behaves just like attr().

   In setter mode, it behaves like attr(), but with the following
   exceptions.

   If the attribute value is a scalar, it'll be only assigned to the first
   element of the set (as ids are supposed to be unique), and the returned
   object will only contain that first element.

       my $first_element = $q->id('the_one');

   It's possible to set the ids of all the elements by passing a sub to
   id(). The sub is given the same arguments as for each(), and its return
   value is taken to be the new id of the elements.

       $q->id( sub { my $i = shift;  'foo_' . $i } );

  name

   Get/set the elements's 'name' attribute.

       my $name = $q->name;  # equivalent to $q->attr( 'name' );

       $q->name( 'foo' );    # equivalent to $q->attr( name => 'foo' );

  data

   Get/set the elements's 'data-*name*' attributes.

       my $data = $q->data('foo');  # equivalent to $q->attr( 'data-foo' );

       $q->data( 'foo' => 'bar' );  # equivalent to $q->attr( 'data-foo' => 'bar' );


 tagname

   Get/Set the tag name of elements.

       my $name = $q->tagname;

       $q->tagname($new_name);

 before

   Insert content, specified by the parameter, before each element in the
   set of matched elements.

       wq('<div><p>foo</p></div>')->find('p')
                                  ->before('<b>bar</b>')
                                  ->end
                                  ->as_html; # <div><b>bar</b><p>foo</p></div>


   The content can be anything accepted by "new".

 clone

   Create a deep copy of the set of matched elements.

 detach

   Remove the set of matched elements from the DOM.

 has_class

   Determine whether any of the matched elements are assigned the given
   class.

  html

   Get/Set the innerHTML.

       my @html = $q->html();

       my $html = $q->html(); # 1st matching element only

       $q->html('<p>foo</p>');

 insert_before

   Insert every element in the set of matched elements before the target.

 insert_after

   Insert every element in the set of matched elements after the target.

  prepend

   Insert content, specified by the parameter, to the beginning of each
   element in the set of matched elements.

 remove

   Delete the elements associated with the object from the DOM.

       # remove all <blink> tags from the document
       $q->find('blink')->remove;

 remove_class

   Remove a single class, multiple classes, or all classes from each
   element in the set of matched elements.

 replace_with

   Replace the elements of the object with the provided replacement. The
   replacement can be a string, a Web::Query object or an anonymous
   function. The anonymous function is passed the index of the current
   node and the node itself (with is also localized as $_).

       my $q = wq( '<p><b>Abra</b><i>cada</i><u>bra</u></p>' );

       $q->find('b')->replace_with('<a>Ocus</a>);
           # <p><a>Ocus</a><i>cada</i><u>bra</u></p>

       $q->find('u')->replace_with($q->find('b'));
           # <p><i>cada</i><b>Abra</b></p>

       $q->find('i')->replace_with(sub{
           my $name = $_->text;
           return "<$name></$name>";
       });
           # <p><b>Abra</b><cada></cada><u>bra</u></p>

 size

   Return the number of elements in the Web::Query object.

       wq('<div><p>foo</p><p>bar</p></div>')->find('p')->size; # 2

 text

   Get/Set the text.

       my @text = $q->text();

       my $text = $q->text(); # 1st matching element only

       $q->text('text');


   If called in a scalar context, only return the string representation of
   the first element

OTHERS

   Web::Query->last_response()

     Returns last HTTP response status that generated by new_from_url().

HOW DO I CUSTOMIZE USER AGENT?

   You can specify your own instance of LWP::UserAgent.

       $Web::Query::UserAgent = LWP::UserAgent->new( agent => 'Mozilla/5.0' );

FAQ AND TROUBLESHOOTING

How to find XML processing instructions in a document?

   It's possible with Web::Query::LibXML and by using an xpath expression
   with find():

       # find <?xml-stylesheet ... ?>
       $q->find(\"//processing-instruction('xml-stylesheet')");

   However, note that the support for processing instructions in
   HTML::TreeBuilder::LibXML::Node is sketchy, so there are methods like
   attr() that won't work.

Can't get the content of script elements

   The <script> tag is treated differently by HTML::TreeBuilder, the
   parser used by Web::Query. To retrieve the content, you can use either
   the method html() (with the caveat that the content will be escaped),
   or use Web::Query::LibXML, which parse the 'script' element
   differently.

       my $node = "<script>var x = '<p>foo</p>';</script>";

       say Web::Query::wq( $node )->text;
           # nothing is printed!

       say Web::Query::wq( $node )->html;
           # var x = &#39;&lt;p&gt;foo&lt;/p&gt;&#39;;

       say Web::Query::LibXML::wq( $node )->text;
           # var x = '<p>foo</p>';

       say Web::Query::LibXML::wq( $node )->html;
           # var x = '&lt;p&gt;foo&lt;/p&gt;';

INCOMPATIBLE CHANGES

   0.10

     new_from_url() is no longer throws exception on bad response from
     HTTP server.

AUTHOR

   Tokuhiro Matsuno <tokuhirom AAJKLFJEF@ GMAIL COM>

SEE ALSO

   pQuery

   XML::LibXML::jQuery

LICENSE

   Copyright (C) Tokuhiro Matsuno

   This library is free software; you can redistribute it and/or modify it
   under the same terms as Perl itself.

You are viewing proxied material from ftp.icm.edu.pl. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.