|
|
hello
i am new to perl.
i need a programm like a "spider" that collects all links from a website (not only the first page) and store them into a file or into a database(access).
is it possible...?...pls help
|
|
|
|
if you need to do it in perl, yes it's possible. One of the easier ways would be to install the module WWW::Mechanize (search for it on search.cpan.org).
You can basically create a WWW:Mechanize object to go to a page and use the $mech->links() function to get a list of links. inserting it into a db should be simple enough depending on what db you plan on using.
semper fi...
|
|
|
|
could help a little more??
maybe an example?
thanx
|
|
|
|
The following is pretty basic (remember you need to install WWW::Mechanize first)
If you want additional examples google is your friend.
Here's one article to get you started though
http://hacks.oreilly.com/pub/h/946
#!/usr/bin/perl
use strict;
# includes
use WWW::Mechanize;
use HTTP::Cookies;
# start
my $client = WWW::Mechanize->new();
$client->get( "http://www.yahoo.com" );
print $client->success();
for my $link ($client->links()) {
print $link->[0];
print "\n";
}
|
|
|
theodore
