Module HawlerHelper
In: lib/hawlerhelper.rb  (CVS)

Methods

Public Class methods

Given some data, find all possible URIs, regardless of whether or not they are actual links

Given a uri, return all possible variations of the path.

Send an HTTP GET request to uri, setting a Referer header of referer, setting other headers from the headers hash. Return the Net::HTTPResponse object

Given a uri and the data returned from retrieving said URI extract all links. Pulls links from the following tag types: a, link, img, script, frame and form. Missing some? Let me know!

Send an HTTP HEAD request to uri, setting a Referer header of referer, setting other headers from the headers hash. Return the Net::HTTPResponse object

Is uri offsite from referer?

Send an HTTP method of method request to uri, setting a Referer header of referer, setting other headers from the headers hash. Return the Net::HTTPResponse object

Is this URI valid? Essentially just calls URI.parse()

[Validate]