Larbin is up and running !

Start date :   Tue Mar 11 11:05:00 2008
Current date : Tue Mar 11 11:11:38 2008
up: 0:06:38

Pages :


urls treated :          9731   (current rate :  34, overall rate :  24)
forb robots :            101
no dns :                 478

Pages :                 9112   (current rate :  33, overall rate :  22)
Success :               2603   (current rate :  10, overall rate :   6)
no connection :           41
early stop :              68
timeout :               3377
bad type :               501
too big :                668
err 30X :                610
err 40X :                593

urls accepted : 123809 / 64000000

fast robots :           1140
fast no conn :             4
fast no dns :            498
too deep :                 0
dup url :              82950

Sites seen (dns call done) :


total number :               7725 +100   (current rate : 23, overall rate : 19)
with dns :                   7407       (current rate : 22, overall rate : 18)
with robots.txt :            1856
with good robots.txt :       1430

sites ready      :  2307
sites without ip :  7019

drwx------ 2 root root 45056 03-11 11:10 d00000
drwx------ 2 root root  4096 03-10 17:43 d00000.ta
drwx------ 2 root root 45056 03-11 11:13 d00001
drwx------ 2 root root 45056 03-11 11:16 d00002
drwx------ 2 root root 45056 03-11 11:19 d00003
drwx------ 2 root root 45056 03-11 11:22 d00004
drwx------ 2 root root 45056 03-11 11:24 d00005
drwx------ 2 root root 45056 03-11 11:27 d00006
drwx------ 2 root root 45056 03-11 11:29 d00007
drwx------ 2 root root 45056 03-11 11:32 d00008
drwx------ 2 root root 45056 03-11 11:34 d00009
drwx------ 2 root root 45056 03-11 11:37 d00010
drwx------ 2 root root 45056 03-11 11:39 d00011
drwx------ 2 root root 45056 03-11 11:42 d00012
drwx------ 2 root root 45056 03-11 11:44 d00013
drwx------ 2 root root 12288 03-11 11:45 d00014

# now, let's customize the search

# first page to fetch (you can specify several urls)
startUrl http://www.ku6.com/

# Do you want to limit your search to a specific domain ?
# if yes, uncomment the following line
#limitToDomain .ku6.com end

                                                      Tue Mar 11 11:11:39 2008