Re: How do you check for broken images if the robot.txt prevents /images/
[
Date Prev][
Date Next]
[
Thread Prev][
Thread Next]
Re: How do you check for broken images if the robot.txt prevents /images/
- From: Arthur de Jong <arthur [at] arthurdejong.org>
- To: webcheck-users [at] lists.arthurdejong.org
- Subject: Re: How do you check for broken images if the robot.txt prevents /images/
- Date: Mon, 04 Jun 2012 19:57:20 +0200
On Sun, 2012-06-03 at 22:20 -0500, Frank Barcenas wrote:
> How do you check for broken images if the robot.txt prevents /images/
You could use the --ignore-robots option to ignore robots.txt files. If
you do need to avoid other stuff in robots.txt you should probably use
the --yank option.
--
-- arthur - arthur@arthurdejong.org - http://arthurdejong.org --
--
To unsubscribe send an email to
webcheck-users-unsubscribe@lists.arthurdejong.org or see
http://lists.arthurdejong.org/webcheck-users/
