lists.arthurdejong.org
RSS feed

Re: How do you check for broken images if the robot.txt prevents /images/

[Date Prev][Date Next] [Thread Prev][Thread Next]

Re: How do you check for broken images if the robot.txt prevents /images/



On Sun, 2012-06-03 at 22:20 -0500, Frank Barcenas wrote:
> How do you check for broken images if the robot.txt prevents /images/

You could use the --ignore-robots option to ignore robots.txt files. If
you do need to avoid other stuff in robots.txt you should probably use
the --yank option.

-- 
-- arthur - arthur@arthurdejong.org - http://arthurdejong.org --
-- 
To unsubscribe send an email to
webcheck-users-unsubscribe@lists.arthurdejong.org or see
http://lists.arthurdejong.org/webcheck-users/