* * * * *

                             A very rare request

I got an email today from someone looking for a tutorial or resources on
writing a metasearch engine. I got the email because I've written three of
them [1] over the past six years or so.

I told the person that I know of no tutorials or references about writing
metasearch engines since when I started, there were about two in existance
that I knew of and that to me, it was pretty straightforward what you need to
do in order to write a metasearch engine: you get query (using CGI most
likely), reformat the queries for each engine you support and make the
request like a browser would (which means you need to support HTTP and CGI
from the client side) and then process the pages you get back (so you need to
parse HTML) and display output like any CGI script can.

Easy.

Okay, maybe not that easy as there are some nagging details you only find out
about by doing an actual implementation (like certain IIS servers will send
out two complete header sections, or that IIS doesn't follow the HTTP
redirect specification at all, and what exactly are you supposed to do if you
get a redirect on a POST? [2]) but if you take it piece by piece it's not
that overwhelming.

Or is it just me?

[1] http://www.conman.org/people/spc/refs/search/
[2] gopher://gopher.conman.org/0Phlog:2001/11/25.1

Email author at [email protected]

You are viewing proxied material from gopher.conman.org. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.