Related references
Note: Only part of the references are listed.
Article
Computer Science, Information Systems
A web crawler design for data mining
M Thelwall
JOURNAL OF INFORMATION SCIENCE (2001)