Web Server Statistics for BMRB mirror webserver

Analyzed requests from Wed, Oct 26 2005 at 5:52 AM to Mon, May 12 2008 at 1:03 AM (928.80 days).

(Go To: Top | Browser Summary | Browser Report)

Browser Summary

Listing browsers with at least 10 requests, sorted by the number of requests.

<
#reqs%reqsGbytesbrowser
263291431.49%392.88MSIE
179345721.45%230.44  MSIE/6
747130 8.93%65.55  MSIE/7
89669 1.07%96.73  MSIE/5
2257 0.03%0.08  MSIE/4
65 0.00  MSIE/3
62 0.00  MSIE/2
37 0.00  MSIE/8
31 0.00  MSIE/999
156447418.71%93.54Netscape
153009118.30%92.61  Netscape/4
28771 0.34%0.63  Netscape/7
3986 0.05%0.27  Netscape/8
1380 0.02%0.03  Netscape/6
117 0.00  Netscape/3
69 0.00  Netscape/0
19 0.00  Netscape/Nutch-0
119701914.31%190.54Firefox
693466 8.29%65.58  Firefox/2
483498 5.78%122.67  Firefox/1
15524 0.19%2.21  Firefox/0
3687 0.04%0.06  Firefox/3
107 0.00  Firefox/9
59 0.00  Firefox/8
597906 7.15%152.81Netscape (compatible)
252759 3.02%3.93WebVac (webmaster@pita.stanford.edu WebVac.org )
232475 2.78%4.96GuideBot
204951 2.45%46.13Safari
94121 1.13%20.49  Safari/419
33564 0.40%21.80  Safari/523
32777 0.39%0.92  Safari/312
18846 0.23%1.83  Safari/417
10239 0.12%0.17  Safari/525
3928 0.05%0.06  Safari/522
3891 0.05%0.13  Safari/416
2950 0.04%0.12  Safari/412
2020 0.02%0.10  Safari/125
1887 0.02%0.48  Safari/85
318 0.00  Safari/420
191 0.00  Safari/100
65 0.00  Safari/413
37 0.00  Safari/51
22 0.00  Safari/5525
21 0.00  Safari/YY
19 0.00  Safari/521
18 0.00  Safari/74
188133 2.25%11.10FAST Enterprise Crawler 6
188133 2.25%11.10  FAST Enterprise Crawler 6 /
173823 2.08%30.92ConveraCrawler
173823 2.08%30.92  ConveraCrawler/0
167802 2.01%31.55msnbot-media
167802 2.01%31.55  msnbot-media/1
133061 1.59%35.61Netscape
102673 1.23%24.53  Mozilla/1
1138 0.01%1.33  Mozilla/0
131212 1.57%13.33Wget
131212 1.57%13.33  Wget/1
74161 0.89%44.11ChemieDE-EF-Bot
74161 0.89%44.11  ChemieDE-EF-Bot/0
67348 0.81%1.20libwww-perl
67348 0.81%1.20  libwww-perl/5
56442 0.67%7.00Opera
38578 0.46%2.96  Opera/9
14117 0.17%3.54  Opera/8
3392 0.04%0.47  Opera/7
283 0.02  Opera/6
44 0.01  Opera/5
12 0.00  Opera/2
54307 0.65%1.26msnbot
36489 0.44%0.90  msnbot/1
17818 0.21%0.36  msnbot/0
41070 0.49%5.29WIRE
41070 0.49%5.29  WIRE/0
37629 0.45%0.50News_Search_App
37629 0.45%0.50  News_Search_App/0
35124 0.42%0.87noxtrumbot
35124 0.42%0.87  noxtrumbot/1
30542 0.37%1.03Java
30542 0.37%1.03  Java/1
28136 0.34%43.21NutchCVS
28091 0.34%43.21  NutchCVS/0
26216 0.31%368.58IRP_edu_bot
26216 0.31%368.58  IRP_edu_bot/Nutch-1
24840 0.30%1.54ichiro
18867 0.23%0.52  ichiro/2
5971 0.07%1.02  ichiro/3
23005 0.28%0.48Baiduspider+(+http:
23005 0.28%0.48  Baiduspider+(+http://www
22297 0.27%0.23Gigabot
19255 0.23%0.19  Gigabot/2
3020 0.04%0.04  Gigabot/3
21356 0.26%0.14Sensis Web Crawler (search_comments\\at\\sensis\\dot\\com\\dot\\au)
19075 0.23%0.22ASPseek
19075 0.23%0.22  ASPseek/1
16027 0.19%0.21OmniWeb
15816 0.19%1.34Konqueror
15265 0.18%1.33  Konqueror/3
537 0.01%0.01  Konqueror/2
13076 0.16%1.77MJ12bot
13076 0.16%1.77  MJ12bot/v1
11589 0.14%0.76Yeti
10630 0.13%0.68  Yeti/0
940 0.01%0.01  Yeti/1
11386 0.14%0.20FAST MetaWeb Crawler (helpdesk at fastsearch dot com)
10101 0.12%0.04UCmore
9053 0.11%20.85bot
9053 0.11%20.85  bot/1
8856 0.11%0.18Ocelli
8856 0.11%0.18  Ocelli/1
8607 0.10%0.25ZoomSpider - wrensoft.com
7790 0.09%2.29MSRBOT (http:
7790 0.09%2.29  MSRBOT (http://research
6753 0.08%0.13Seeker.lookseek.com
6743 0.08%0.73Python-urllib
4958 0.06%0.45  Python-urllib/1
1785 0.02%0.28  Python-urllib/2
6557 0.08%17.51bisdann
6413 0.08%17.48  bisdann/Nutch-0
144 0.04  bisdann/Nutch-1
5910 0.07%0.72Camino
5624 0.07%0.71  Camino/1
286 0.01  Camino/0
5856 0.07%0.42lwp-trivial
5856 0.07%0.42  lwp-trivial/1
5676 0.07%0.11Accoona-AI-Agent
5676 0.07%0.11  Accoona-AI-Agent/1
5612 0.07%0.57larbin_2.6.3 larbin2.6.3@unspecified.mail
5442 0.07%0.14LWP::Simple
5442 0.07%0.14  LWP::Simple/5
5158 0.06%0.24Jakarta Commons-HttpClient
4983 0.06%0.23  Jakarta Commons-HttpClient/3
175 0.00  Jakarta Commons-HttpClient/2
4856 0.06%0.17Lynx
4854 0.06%0.17  Lynx/2
4360 0.05%0.17Irvine
4360 0.05%0.17  Irvine/1
4050 0.05%0.11RufusBot (Rufus Web Miner; http:
4044 0.05%0.11  RufusBot (Rufus Web Miner; http://64
3833 0.05%0.11Java1.3.1
3426 0.04%10.66ilial
3426 0.04%10.66  ilial/Nutch-0
3204 0.04%1.18CazoodleBot
1503 0.02%1.15  CazoodleBot/CazoodleBot-0
1389 0.02%0.02  CazoodleBot/Nutch-0
312 0.00  CazoodleBot/0
3058 0.04%0.07TurnitinBot
3058 0.04%0.07  TurnitinBot/2
2995 0.04%0.41ia_archiver
2956 0.04%0.22Java1.3.1_16
2785 0.03%0.17Java1.3.1_18
2633 0.03%0.06lanshanbot
2633 0.03%0.06  lanshanbot/1
2497 0.03%0.01arianna.libero.it
2350 0.03%10.16CFNetwork
1577 0.02%8.23  CFNetwork/129
251 1.92  CFNetwork/4
246 0.00  CFNetwork/221
178 0.00  CFNetwork/220
48 0.00  CFNetwork/217
37 0.00  CFNetwork/10
12 0.00  CFNetwork/1
2260 0.03%0.67sproose
2058 0.02%0.67  sproose/1
202 0.00  sproose/0
2246 0.03%18.93aipbot
2157 0.03%16.24  aipbot/1
89 2.70  aipbot/2
2182 0.03%0.07Java1.3.1_02
2021 0.02%0.02msnbot-Products
2021 0.02%0.02  msnbot-Products/1
2014 0.02%0.01voyager
2014 0.02%0.01  voyager/1
1972 0.02%0.04W3CRobot
1972 0.02%0.04  W3CRobot/5
1951 0.02%0.09halloteser
1951 0.02%0.09  halloteser/Nutch-0
1791 0.02%0.02Yandex
1791 0.02%0.02  Yandex/1
1785 0.02%0.09FAST-WebCrawler
1764 0.02%0.09  FAST-WebCrawler/3
21 0.00  FAST-WebCrawler/2
1783 0.02%0.09Teemer (NetSeer, Inc. is a Los Angeles based Internet startup company.; http:
1783 0.02%0.09  Teemer (NetSeer, Inc. is a Los Angeles based Internet startup company.; http://www
1761 0.02%0.04edu-search
1731 0.02%6.22MultiText
1731 0.02%6.22  MultiText/0
1470 0.02%0.02WinHTTP Robot
1470 0.02%0.02  WinHTTP Robot/1
1466 0.02%0.03woriobot (+http:
1466 0.02%0.03  woriobot (+http://www
1399 0.02%0.01appie
1399 0.02%0.01  appie/1
1382 0.02%0.01Internet Explorer 6.0
1352 0.02%0.01envolk
1352 0.02%0.01  envolk/1
1295 0.02%0.01RedBot
1295 0.02%0.01  RedBot/redbot-1
1250 0.01%5.69DoIT Search Server (Beta)
1250 0.01%5.69  DoIT Search Server (Beta)/0
1247 0.01%0.18gnu-classpath
1241 0.01%0.18  gnu-classpath/0
1226 0.01%0.07Java1.3.1_15
1193 0.01%0.04hl_ftien_spider_v1.1
1184 0.01%0.03192.comAgent
1161 0.01%0.02egothor
1159 0.01%0.02  egothor/7
1160 0.01%0.03nicebot
1154 0.01%0.03Shim-Crawler(Mozilla-compatible; http:
1154 0.01%0.03  Shim-Crawler(Mozilla-compatible; http://www
1024 0.01%0.02Galeon
968 0.01%0.02  Galeon/1
56 0.00  Galeon/2
1007 0.01%0.02disco
560 0.01%0.01  disco/Nutch-1
447 0.01%0.01  disco/Nutch-0
974 0.01%0.08Java1.3.1_17
966 0.01%0.02Java1.2.2
936 0.01%0.01Microsoft URL Control - 6.00.8862
912 0.01%2.83MLBot (www.metadatalabs.com)
854 0.01%0.01psycheclone
849 0.01%0.00DataCha0s
849 0.01%0.00  DataCha0s/2
823 0.01%0.02TsWeb
823 0.01%0.02  TsWeb/Nutch-0
811 0.01%0.01webcollage
811 0.01%0.01  webcollage/1
772 0.01%0.00Speedy Spider (http:
772 0.01%0.00  Speedy Spider (http://www
712 0.01%0.01Factbot
712 0.01%0.01  Factbot/1
707 0.01%0.00findlinks
706 0.01%0.00  findlinks/1
702 0.01%0.01SBIder
702 0.01%0.01  SBIder/0
683 0.01%0.01MQBOT
683 0.01%0.01  MQBOT/Nutch-0
681 0.01%0.04Java1.3.1_09
679 0.01%1.34PEAR HTTP_Request class ( http:
679 0.01%1.34  PEAR HTTP_Request class ( http://pear
674 0.01%0.03genieBot wgao@genieknows.com
668 0.01%0.01POE-Component-Client-HTTP
668 0.01%0.01  POE-Component-Client-HTTP/0
660 0.01%0.00facebookexternalhit
660 0.01%0.00  facebookexternalhit/1
637 0.01%0.01dragonfly(ebingbong@playstarmusic.com)
630 0.01%0.01MSNPTC
630 0.01%0.01  MSNPTC/1
629 0.01%0.01411.info WebBot
629 0.01%0.01  411.info WebBot/0
623 0.01%0.01HttpClient
622 0.01%0.18spider tspyyp@tom.com
579 0.01%0.02Java1.1.8
571 0.01%0.00NaverBot-1.0 (NHN Corp.
571 0.01%0.00  NaverBot-1.0 (NHN Corp. /
548 0.01%0.01Nutch0.8
548 0.01%0.01  Nutch0.8/Nutch-0
523 0.01%0.00FeedFetcher-Google; (+http:
523 0.01%0.00  FeedFetcher-Google; (+http://www
517 0.01%0.01boitho.com-dc
517 0.01%0.01  boitho.com-dc/0
509 0.01%0.01pixfinder
509 0.01%0.01  pixfinder/2
471 0.01%0.00OmniExplorer_Bot
463 0.01%0.00  OmniExplorer_Bot/6
461 0.01%0.01OutfoxBot
461 0.01%0.01  OutfoxBot/0
460 0.01%0.01Microsoft URL Control - 6.01.9782
456 0.01%0.00OpenIntelligenceData
456 0.01%0.00  OpenIntelligenceData/1
451 0.01%0.01worio heritrix bot (+http:
451 0.01%0.01  worio heritrix bot (+http://worio
450 0.01%0.01(null)
447 0.01%0.01larbin_2.6.3 gqnmgsp@ruc.edu.cn
433 0.01%0.01ksibot
433 0.01%0.01  ksibot/8
430 0.01%0.02ZipppBot
430 0.01%0.02  ZipppBot/0
422 0.01%0.01complex_network_group
422 0.01%0.01  complex_network_group/Nutch-0
421 0.01%0.00http:
413 0.00  http://www
415 1.11Tasapspider
415 1.11  Tasapspider/0
413 0.00OCP HRS 1.0
413 0.04larbin_2.6.3 atat@cs.bilkent.edu.tr
406 0.00ia_archiver-web.archive.org
400 0.00Szukacz
400 0.00  Szukacz/1
399 0.00Avant Browser (http:
399 0.00  Avant Browser (http://www
397 0.06yacybot (amd64 Linux 2.6.22-14-generic; java 1.5.0_13; GMT+01:00
397 0.06  yacybot (amd64 Linux 2.6.22-14-generic; java 1.5.0_13; GMT+01:00/en)
391 0.00Speedy Spider (Entireweb; Beta
391 0.00  Speedy Spider (Entireweb; Beta/1
389 0.03Java1.3.1_04
387 0.00Microsoft Data Access Internet Publishing Provider Protocol Discovery
377 0.00Snapbot
377 0.00  Snapbot/1
370 0.01EnaBot
370 0.01  EnaBot/1
365 0.00larbin_2.6.3 marzia.polito@intel.com
362 0.00Balihoo
362 0.00  Balihoo/Nutch-1
362 0.00Spider
362 0.00nrsbot
362 0.00  nrsbot/5
359 0.00VisBot
353 0.00  VisBot/2
356 0.00bmrbFileGetSend.java code
348 0.17Gaisbot
348 0.17  Gaisbot/3
346 0.00cfetch
346 0.00  cfetch/1
338 0.00LinkScan
214 0.00  LinkScan/8
124 0.00  LinkScan/11
335 0.11Firebird
335 0.11  Firebird/0
335 0.00LocalcomBot
335 0.00  LocalcomBot/1
334 0.12Sosospider+(+http:
334 0.12  Sosospider+(+http://help
334 0.01Twiceler www.cuill.com
326 0.00  Twiceler www.cuill.com/robots
332 1.43WebSpider.vbs
330 0.00genieBot (http:
330 0.00  genieBot (http://64
329 0.00HouxouCrawler
327 0.00  HouxouCrawler/Nutch-0
328 0.00AgentName
328 0.00  AgentName/0
327 0.02Java1.3.1_13
324 0.00Xenu Link Sleuth 1.2g
307 0.00DoCoMo
298 0.00  DoCoMo/2
303 0.00larbin_2.6.3 (larbin2.6.3@unspecified.mail)
300 0.00envolk[ITS]spider
300 0.00  envolk[ITS]spider/1
299 0.00occsSiteChecker
295 0.00DataFountains
271 0.00  DataFountains/DMOZ
24 0.00  DataFountains/Dmoz
292 0.00Sleipnir
218 0.00  Sleipnir/2
289 0.00Web Downloader
286 0.00  Web Downloader/7
289 1.39beta.spider@gmail.com
287 0.44WebCorp
287 0.44  WebCorp/1
282 78.06FDM
269 78.05  FDM/1
13 0.01  FDM/2
275 0.00NetResearchServer
264 0.00  NetResearchServer/4
11 0.00  NetResearchServer/3
275 0.00My agent
273 0.01Oracle Secure Enterprise Search
272 0.11Goldfire Server
270 0.03Teemer (NetSeer, Inc.; http:
270 0.03  Teemer (NetSeer, Inc.; http://www
267 0.00MQbot metaquerier.cs.uiuc.edu
267 0.00  MQbot metaquerier.cs.uiuc.edu/crawler
263 0.01Java1.3.1_03
261 0.00IU_CSCI_B659_class_crawler
261 0.00  IU_CSCI_B659_class_crawler/1
258 0.00FAST Enterprise Crawler 6 used by Noreply (noreply@noreply.com)
254 0.00UdmSearch
254 0.00  UdmSearch/3
253 0.00DTAAgent
251 0.00BioNutchCVS
251 0.00  BioNutchCVS/0
250 0.00iaskspider
244 0.00  iaskspider/2
246 0.00CCBot
246 0.00  CCBot/1
244 0.00findfiles.org
244 0.00  findfiles.org/0
244 0.00Pagebull http:
244 0.00  Pagebull http://www
239 0.00WordPress
173 0.00  WordPress/2
51 0.00  WordPress/MU
235 0.00cshopper
235 0.00  cshopper/Nutch-1
235 0.00Missigua Locator 1.9
235 0.00LinkLint-checkonly
235 0.00  LinkLint-checkonly/2
232 0.01SpiderMan
229 0.00Nutch
225 0.00  Nutch/Nutch-0
225 0.00Watchfire WebXM
222 0.011.0
221 0.00Bigsearch.ca
121 0.00  Bigsearch.ca/Nutch-0
100 0.00  Bigsearch.ca/Nutch-1
217 0.00Nutch Test
217 0.00  Nutch Test/Something
208 0.00dragonfly(ebingbong#playstarmusic.com)
206 0.00FAST Enterprise Crawler 6 used by Singapore Press Holdings (crawler@sphsearch.sg)
197 0.00genieBot genieBot@genieknows.com
187 0.01Java1.3.1_01
180 0.01MSR-ISRCCrawler
178 0.01OSSProxy
178 0.01  OSSProxy/1
176 0.00ISC Systems iRc Search 2.1
174 0.00ScientificCommons.org
174 0.00  ScientificCommons.org/2
173 0.00Avant Browser
173 0.00Alpha Search Agent
172 0.00Uni-Hannover-Indexer-UnisUSA
171 0.00nutch
149 0.00  nutch/Nutch-1
12 0.00  nutch/Nutch-0
10 0.00  nutch/0
158 0.00Microsoft URL Control - 6.00.8169
157 4.41NPBot
157 4.41  NPBot/3
156 0.00Agent Anonymous
156 0.00  Agent Anonymous/3G
152 0.00Grub
152 0.00  Grub/2
152 0.00Semager
152 0.00  Semager/1
149 0.03yacybot (i386 Linux 2.6.22-14-generic; java 1.5.0_13; Europe
149 0.03  yacybot (i386 Linux 2.6.22-14-generic; java 1.5.0_13; Europe/en)
147 0.00Squid-Prefetch
146 0.00WebAlta Crawler
146 0.00  WebAlta Crawler/2
144 0.00GreenBrowser
143 0.02Java(tm) 2 SDK, Standard Edition v1.7.0 Java
143 0.02  Java(tm) 2 SDK, Standard Edition v1.7.0 Java/1
142 0.07BDFetch
142 0.00snap.com beta crawler v0
139 0.00yggdrasil
139 0.00  yggdrasil/Nutch-0
138 0.00Java1.3.0
136 0.01Java1.1.6
135 0.00favorstarbot
135 0.00  favorstarbot/1
135 0.50sogou spider
131 0.00g2Crawler nobody@airmail.net
129 0.01Java1.3.1_06
127 0.06curl
127 0.06  curl/7
126 0.00Haansoft
124 0.00MateiSpider
124 0.00  MateiSpider/Nutch-0
122 0.01Java1.3.1_10
120 0.00SiteBar
120 0.00  SiteBar/3
119 0.00link_checker
105 0.00  link_checker/1
117 0.00Xenu Link Sleuth 1.2i
116 0.00LinkWalker
26 0.00  LinkWalker/2
116 0.00Seeker.lookseek.com see layanegg.com EDU-BOT
112 0.00Xenu Link Sleuth 1.2j
112 0.00Mail.Ru
112 0.00  Mail.Ru/1
111 0.00sohu agent
111 0.00VadixBot
110 0.00w3m
110 0.00  w3m/0
108 0.00category-test-spider
108 0.00  category-test-spider/Nutch-0
108 0.00pulseBot (pulse Web Miner)
106 0.00VSynCrawler
106 0.00  VSynCrawler/1
104 0.00MSIE is
103 0.00msrabot
103 0.00  msrabot/1
103 0.00Clustera Crawler
103 0.00  Clustera Crawler/Nutch-1
102 0.00Libby_1.1
102 0.00  Libby_1.1/libwww-perl/5
100 0.01Java1.3.1_11
100 0.00semCrawler
98 0.00Shim-Crawler
97 0.00telnet [ja] solaris
97 0.00Microsoft Office Protocol Discovery
95 0.00Teleport Pro
95 0.00  Teleport Pro/1
95 0.00education portal
95 0.00  education portal/Nutch-0
94 0.00Nutch-test
94 0.00  Nutch-test/Nutch-0
93 0.00W3C-checklink
93 0.00  W3C-checklink/4
93 0.00IE
85 0.00  IE/6
92 0.00libcurl-agent
92 0.00  libcurl-agent/1
92 0.00CydralSpider
84 0.00  CydralSpider/3
92 0.00Big Brother (http:
92 0.00  Big Brother (http://pauillac
91 0.00Checkbot
91 0.00  Checkbot/1
91 0.00ccubee
55 0.00  ccubee/3
28 0.00  ccubee/4
91 0.00Findexa Crawler (http:
91 0.00  Findexa Crawler (http://www
90 0.00XXX
90 0.18Hey Im not a CC bot
90 0.00Xenu Link Sleuth 1.2h
89 0.00research-spider
89 0.00JoeDog
89 0.00  JoeDog/1
88 0.00WebarooBot (Webaroo Bot; http:
67 0.00  WebarooBot (Webaroo Bot; http://64
21 0.00  WebarooBot (Webaroo Bot; http://www
88 0.00wwwster
88 0.00  wwwster/1
88 0.00WinBatch Internet Extender Ver: 25074
88 0.00gsa-crawler (Enterprise; GIX-03119; misimmon@utmb.edu,tiholden@utmb.edu)
87 0.00great-plains-web-spider
76 0.00  great-plains-web-spider/flatlandbot
11 0.00  great-plains-web-spider/gpws
87 0.00Webcheck
87 0.00  Webcheck/1
84 0.00Biozilla
84 0.00  Biozilla/5
83 0.00Forschungsportal
83 0.00  Forschungsportal/0
82 0.00Infoseek SideWinder
82 0.00  Infoseek SideWinder/2
81 0.00Phoenix
73 0.00  Phoenix/0
81 0.00Pluggd
81 0.00  Pluggd/Nutch-0
80 0.00GurujiBot
80 0.00  GurujiBot/1
80 0.00iVia
66 0.00  iVia/4
14 0.00  iVia/5
79 0.00Exploder
79 0.00  Exploder/0
78 0.00Visbot
78 0.00  Visbot/1
78 0.00REBI-Shoveler v0.1
76 0.00Sogou Orion spider
76 0.00  Sogou Orion spider/3
75 0.00Xenu Link Sleuth 1.2i beta
75 0.00Sosoimagespider+(+http:
75 0.00  Sosoimagespider+(+http://help
72 0.00MSIE enviable
72 0.00T-Online Browser
70 0.00Microschuft
70 0.00  Microschuft/1
70 0.00Pete-Spider Light
70 0.00  Pete-Spider Light/1
70 0.00Inktomi Search
69 0.01BilgiBot
69 0.01  BilgiBot/1
69 0.00Microsoft-WebDAV-MiniRedir
62 0.00  Microsoft-WebDAV-MiniRedir/5
68 0.00MyIE
66 0.00EasyDL
66 0.00  EasyDL/3
65 0.00WebTV
63 0.00  WebTV/2
65 0.00sogou blog spider(+http:
65 0.00  sogou blog spider(+http://www
63 0.00wish-project (http:
63 0.00  wish-project (http://wish
62 0.00IEAutoDiscovery
61 0.00genieBot (wgao@genieknows.com)
60 0.00Nusearch Spider (www.nusearch.com)
60 0.00Java1.3.1_08
60 0.00LinkCheck Scanner
60 0.00  LinkCheck Scanner/0
60 0.00multicrawler (+http:
60 0.00  multicrawler (+http://sw
60 0.01holmes
60 0.01  holmes/3
59 0.00genieBot ((http:
59 0.00  genieBot ((http://64
58 0.00Java1.3.1_12
58 0.00webcollage.perl
58 0.00  webcollage.perl/1
58 0.00TREX
58 0.00Java1.3.0_01
58 0.00Java1.3.0_02
57 0.00SumeetBot (Sumeet Bot; http:
57 0.00  SumeetBot (Sumeet Bot; http://64
57 0.00ejupiter.com
56 0.00.
56 0.00  ./
55 0.00NutchSIS
55 0.00  NutchSIS/Nutch-0
54 0.00Nokia6820
54 0.00  Nokia6820/2
54 0.00ht:
54 0.00  ht://check/1
54 0.00HyperEstraier
54 0.00  HyperEstraier/1
54 0.00Giant
54 0.00  Giant/1
54 0.00Test
54 0.00webGobbler
54 0.00  webGobbler/1
53 0.00New search bot
53 0.00  New search bot/1
53 0.00NASA Search 1.0
53 0.00contype
53 0.00PrivacyFinder
53 0.00  PrivacyFinder/1
53 0.00Krugle
53 0.00  Krugle/Krugle,Nutch/0
52 0.00iVia Page Fetcher (http:
52 0.00  iVia Page Fetcher (http://ivia
51 0.00VB Project
51 0.00SlimBrowser
51 0.00ee:
51 0.00  ee://aol/http
50 0.00Combine
45 0.00  Combine/2
49 0.00Blaiz-Bee
49 0.00  Blaiz-Bee/2
48 0.00Argus
48 0.00  Argus/1
48 0.00Java1.3.1_19
48 0.00g3.pl
48 0.00  g3.pl/0
48 0.00AnswerBus (http:
48 0.00  AnswerBus (http://www
48 0.00Tcl http client package 2.0
47 0.00url checker: biome.ac.uk
47 0.00HTMLParser
46 0.00  HTMLParser/1
46 0.00sna-0.0.1 mikeelliott@hotmail.com
46 0.00BlackBerry8100
46 0.00  BlackBerry8100/4
46 0.00PHP
28 0.00  PHP/5
10 0.00  PHP/4
45 0.18NetAnts
45 0.18  NetAnts/1
44 0.00EI
44 0.00Snoopy v1.2
43 0.00YodaoBot
43 0.00  YodaoBot/1
42 0.00eBingBong - next generation search - ebingbong@playstarmusic.com
42 0.00FAST Enterprise Crawler 6 used by Proctor and Gamble (sales@fastsearch.com)
42 0.00Mozdex
42 0.00  Mozdex/0
42 0.01yacybot (amd64 Linux 2.6.22-14-generic; java 1.6.0_03; Europe
41 0.01  yacybot (amd64 Linux 2.6.22-14-generic; java 1.6.0_03; Europe/en)
41 0.00NG-Search
41 0.00  NG-Search/0
41 0.00yodaoice
41 0.00JoBo
41 0.00  JoBo/1
41 0.00yacy (www.yacy.net; v20040602; amd64 Linux 2.6.22-14-generic; java 1.5.0_13; GMT+01:00
41 0.00  yacy (www.yacy.net; v20040602; amd64 Linux 2.6.22-14-generic; java 1.5.0_13; GMT+01:00/en)
41 0.00JS-UZUSHIO
40 0.00updated
40 0.00  updated/0
39 0.00exooba
39 0.00  exooba/exooba
39 0.00workload-generator
39 0.00  workload-generator/Nutch-0
39 0.04DA
36 0.04  DA/7
39 0.00wish
39 0.00PHP version tracker (http:
39 0.00  PHP version tracker (http://www
38 0.00WebCopier v4.3
38 0.00Robotgenius crawler
38 0.00  Robotgenius crawler/Nutch-1
38 0.00Spock Crawler (http:
38 0.00  Spock Crawler (http://www
38 0.29MyApp
34 0.02  MyApp/0
37 0.00genieBot enash@genieknows.com
37 0.00kalooga
37 0.00  kalooga/kalooga-4
37 0.00Java1.3.1_07
37 0.00TinEye
36 0.00  TinEye/1
37 0.00gnome-vfs
37 0.00  gnome-vfs/2
36 0.00yoofind
36 0.00  yoofind/yoofind-0
36 0.00NutchEC2Test
36 0.00  NutchEC2Test/Nutch-0
36 0.00NavissoBot
17 0.00  NavissoBot/1
36 0.00Googlebot
34 0.00  Googlebot/2
35 0.00IRLbot
35 0.00  IRLbot/2
35 0.00BabalooSpider
35 0.00  BabalooSpider/1
35 0.00CheckLinks
35 0.00  CheckLinks/1
34 0.00web2express.org
34 0.00  web2express.org/Nutch-0
34 0.00sogou develop spider
34 0.00larbin_2.6.3 (gqnmgsp@ruc.edu.cn)
33 0.00Java1.3.1_14
33 0.00responser_mod.pl
33 0.00  responser_mod.pl/0
33 0.00i1searchbot
33 0.00  i1searchbot/2
33 0.00Wells Search II
33 0.00libiViaCore
33 0.00  libiViaCore/5
33 0.00VMBot
33 0.00  VMBot/0
32 0.00MQbot http:
32 0.00  MQbot http://metaquerier
32 0.00Jyxobot
32 0.00  Jyxobot/1
32 0.00gsa-crawler (Enterprise; S5-M43B9PN97SNAS; rajiv@neu.edu)
32 0.00yacybot (i386 Linux 2.6.22-14-generic; java 1.6.0_03; Europe
32 0.00  yacybot (i386 Linux 2.6.22-14-generic; java 1.6.0_03; Europe/en)
31 0.00Seekbot
31 0.00  Seekbot/1
31 0.00FAST Enterprise Crawler 6 used by Comperio AS (sts@comperio.no)
31 3.15iGetter
31 3.15  iGetter/2
31 0.00VWBOT
31 0.00  VWBOT/Nutch-0
30 0.61GetRightPro
30 0.61  GetRightPro/6
30 0.00Labhoo+(+http:
30 0.00  Labhoo+(+http://www
30 0.00SeekGen
30 0.00  SeekGen/Nutch-0
30 0.00eApolloBot
30 0.00  eApolloBot/1
30 0.00Rexa
30 0.00  Rexa/0
30 0.00Poirot
30 0.00csci_b659
30 0.00  csci_b659/0
29 0.00gsa-crawler (Enterprise; MID-02848; support@throttlenet.com)
29 0.00ICC-Crawler(Mozilla-compatible; http:
29 0.00  ICC-Crawler(Mozilla-compatible; http://kc
29 0.00FAST Enterprise Crawler 6 used by EPFL (raul.rabionet@epfl.ch)
29 0.00Crawler.MyCrawler
29 0.48Download Master
28 0.00SpiderKU
28 0.00  SpiderKU/0
28 0.00ozelot
28 0.00  ozelot/2
28 0.00Crawler$2
28 0.00FavOrg
27 0.00PycURL
27 0.00  PycURL/7
27 0.00flatlandbot
27 0.00  flatlandbot/flatlandbot
27 0.00BlackBerry8310
27 0.00  BlackBerry8310/4
27 0.00IIITBOT
27 0.00  IIITBOT/1
27 0.00Xenu Link Sleuth 1.2j (beta)
27 0.00yacybot (amd64 Linux 2.6.22-14-generic; java 1.6.0; Europe
27 0.00  yacybot (amd64 Linux 2.6.22-14-generic; java 1.6.0; Europe/de)
26 0.00HSFT - LVU Scanner
26 0.00larbin_2.6.3 (atat@cs.bilkent.edu.tr)
26 0.00HMSE_Robot
26 0.00Dillo
26 0.00  Dillo/0
26 0.00GOFORITBOT ( http:
26 0.00  GOFORITBOT ( http://www
26 0.00LarbinWebCrawler internet@bredband.net
26 0.00WWW-Mechanize
25 0.00  WWW-Mechanize/1
26 0.00mozilla
21 0.00  mozilla/5
26 0.00yacybot (i386 Mac OS X 10.4.11; java 1.5.0_07; Europe
26 0.00  yacybot (i386 Mac OS X 10.4.11; java 1.5.0_07; Europe/de)
26 0.00silk
26 0.00  silk/1
25 0.00eagle
25 0.00ScienceTool Crawler
25 0.00focused_crawler (flexo; Linux x86_64; http:
25 0.00  focused_crawler (flexo; Linux x86_64; http://ivia
25 0.00ShopWiki
25 0.00  ShopWiki/1
25 0.00sna-0.0.1 (mikeelliott@hotmail.com)
25 0.00urllist
25 0.00  urllist/0
25 0.00Healthbot
25 0.00  Healthbot/Health_and_Longevity_Project_(HealthHaven
24 0.00COMBINE
24 0.00  COMBINE/0
24 0.00Java1.1.7B
24 0.00MaSagool
18 0.00  MaSagool/1
24 0.00Java1.3.1_20
24 0.00vetseek
24 0.00  vetseek/0
24 0.01Java(tm) 2 SDK, Standard Edition v1.6.0 Java
24 0.01  Java(tm) 2 SDK, Standard Edition v1.6.0 Java/1
24 0.00combine
24 0.00  combine/0
24 0.00IlTrovatore-Setaccio
24 0.00  IlTrovatore-Setaccio/1
24 0.00TerrawizBot
24 0.00  TerrawizBot/1
23 0.00IP*Works! V5 HTTP
23 0.00  IP*Works! V5 HTTP/S
23 0.00MSRBOT
23 0.00User-Agent: BoardReader-Image-Fetcher
23 0.00  User-Agent: BoardReader-Image-Fetcher /1
23 0.00WebCopier v3.3
23 0.00LinkChecker
14 0.00  LinkChecker/4
23 0.00WebScouter
23 0.00  WebScouter/1
22 0.00BlackBerry8830
22 0.00  BlackBerry8830/4
22 0.00sohu-search
22 0.00spider (tspyyp@tom.com)
22 0.00KDDI-SA31 UP.Browser
22 0.00  KDDI-SA31 UP.Browser/6
22 0.00VSE
22 0.00  VSE/1
22 0.00LinkScan Server
22 0.00  LinkScan Server/8
22 0.13PageBitesHyperBot
22 0.13  PageBitesHyperBot/600
22 0.00google
22 0.00BlackBerry8700
22 0.00  BlackBerry8700/4
21 0.00yacybot (amd64 Linux 2.6.22-14-generic; java 1.6.0_03; GMT
21 0.00  yacybot (amd64 Linux 2.6.22-14-generic; java 1.6.0_03; GMT/en)
21 0.00Foobzilla
21 0.00  Foobzilla/1
21 0.00Robozilla
21 0.00  Robozilla/1
20 0.09User-Agent: NULL
20 0.00kmaHTTP
20 0.00  kmaHTTP/1
20 0.00HTTP
20 0.00  HTTP/1
20 0.00TestCrawler
20 0.00  TestCrawler/Nutch-0
20 0.00sait
20 0.00  sait/Nutch-0
20 0.00Xenu Link Sleuth 1.2b
20 0.00srSpider larbin2.6.3@unspecified.mail
19 0.00culsearch
19 0.00  culsearch/culs/0
19 0.00Lunascape
19 0.00SimplePie
19 0.00  SimplePie/1
19 0.00Spinne
19 0.00  Spinne/2
18 0.00Nutscrape
18 0.00  Nutscrape/1
18 0.00Enigma Browser
18 0.00NG
18 0.00  NG/2
18 0.00BaiDuSpider
18 0.00SquidClamAV_Redirector 1.6.3
18 0.00Vagabondo
18 0.00  Vagabondo/3
18 0.00Snoopy v1.2.3
17 0.00ScanWebBot
17 0.00  ScanWebBot/1
17 0.00yacy
17 0.00  yacy/0
17 0.00PythonWikipediaBot
17 0.00  PythonWikipediaBot/1
17 0.00zedzo.digest
17 0.00  zedzo.digest/0
17 0.00HTTPConnect
17 0.00IBM Rational AppScan Enterprise
17 0.00page_verifier http:
17 0.00  page_verifier http://www
17 0.00HPL
17 0.00  HPL/Nutch-0
17 0.00TMCrawler
16 0.00Sirketcebot
16 0.00  Sirketcebot/v
16 0.00htdig
12 0.00  htdig/3
16 0.00Digger
16 0.00  Digger/2
16 0.00ResearchBot
16 0.00  ResearchBot/Nutch-0
16 0.00Lexxe
16 0.00  Lexxe/Robot
16 0.00void
16 0.00  void/1
16 0.00Sogou Push Spider
16 0.00  Sogou Push Spider/3
16 0.00testNutch
16 0.00  testNutch/Nutch-0
15 0.00InsumaScout
15 0.00  InsumaScout/1
15 0.00DepSpid - the dependency spider
15 0.00PRCrawler
15 0.00  PRCrawler/Nutch-0
15 0.00BlackBerry8800
15 0.00  BlackBerry8800/4
15 0.00SpaceBison
15 0.00  SpaceBison/0
15 0.00ozzie (question ozzie@cs.stanford.edu)
15 0.00autowebdir
15 0.00  autowebdir/1
15 0.00Zeusbot
15 0.00  Zeusbot/0
15 0.00MOMspider
15 0.00  MOMspider/1
15 0.00Crawllybot
15 0.00  Crawllybot/0
15 0.00Crawler0.1
15 0.00wish-lim
15 0.00Pita (webmaster@pita.stanford.edu)
15 0.00HooWWWer
15 0.00  HooWWWer/2
14 0.00Uoi-Adzap-Agent
14 0.00  Uoi-Adzap-Agent/9
14 0.00SuperBot
14 0.00  SuperBot/4
14 0.00WebTrends Link Analyzer
14 0.00Perl-LWP
14 0.00  Perl-LWP/0
14 0.00Microsoft Data Access Internet Publishing Provider Cache Manager
14 0.00AOL_Ian_The_Aussie_Spider
14 0.00  AOL_Ian_The_Aussie_Spider/Nutch-0
14 0.00ShablastBot
14 0.00  ShablastBot/1
14 0.00onCHECK-Robot, www.onsearch.de
14 0.00iVia Downloader
14 0.00Advanced Browser (http:
14 0.00  Advanced Browser (http://www
14 0.00google_refuse_wget
14 0.00gsa-crawler (Enterprise; GIX-03055; mark.roberts@iraq.centcom.mil)
13 0.00Sigram
13 0.00NutchCVS (Nutch; http:
13 0.00  NutchCVS (Nutch; http://lucene
13 0.00simple_get
13 0.00  simple_get/1
13 0.00Microsoft Office Existence Discovery
13 0.00MSIE incompatible
13 0.00gFTP
13 0.00  gFTP/2
13 0.00Lsearch
13 0.00  Lsearch/sondeur
13 0.00CASANOVA0.1
13 0.00BlackBerry8703e
13 0.00  BlackBerry8703e/4
13 0.00yacybot (i386 Linux 2.6.22-14-generic; java 1.5.0_13; GMT+01:00
13 0.00  yacybot (i386 Linux 2.6.22-14-generic; java 1.5.0_13; GMT+01:00/en)
12 0.00*
12 0.00  */Nutch-0
12 0.00HPPrint
12 0.00nnn
12 0.00  nnn/ttt
12 0.00SonyEricssonK310i
12 0.00  SonyEricssonK310i/R4EA
12 0.00GiveALink
12 0.00vspider
12 0.00wget
12 0.00Firefox Firefox
12 0.00MetagerBot
12 0.00  MetagerBot/0
12 0.08RMA
12 0.08  RMA/1
12 0.00AdSenserBot
12 0.00caizhi
12 0.00  caizhi/infomation/Nutch-0
12 0.00Microsoft Internet Explorer
12 0.00Taiga web spider
12 0.00alef
12 0.00  alef/0
12 0.00MultiCrawler, juergen.umbrich@deri.org
12 0.00Cazoodle
12 0.00  Cazoodle/Nutch-0
12 0.00BlackBerry8300
12 0.00  BlackBerry8300/4
12 0.00URLBase
12 0.00  URLBase/6
12 0.00EMPAS_ROBOT
12 0.00healia
12 0.00  healia/Nutch-0
12 0.00JACK-O`-LANTERN
12 0.00  JACK-O`-LANTERN/1
12 0.00Netscape-5.0
12 0.00WORDIAN Document Filter
12 0.00Jambot
12 0.00  Jambot/0
12 0.00BlackBerry7250
12 0.00  BlackBerry7250/4
11 0.00FAST Enterprise Crawler 6 used by FAST (sales@fastsearch.com)
11 0.00FAST Enterprise Crawler 6 used by Search (search@search.com)
11 0.00Microsoft Data Access Internet Publishing Provider DAV 1.1
11 0.00zedzo.validate
11 0.00  zedzo.validate/0
11 0.00larbin_2.6.3 (marzia.polito@intel.com)
11 0.00oBot
11 0.00GT::WWW
11 0.00  GT::WWW/1
11 0.00Activator
11 0.00  Activator/1
11 0.00netscape
11