webcheck commit: r471 - webcheck/webcheck

[Commits of the webcheck project <webcheck-commits [at] lists.arthurdejong.org>]

Author: arthur
Date: Wed Aug 29 17:34:23 2012
New Revision: 471
URL: http://arthurdejong.org/viewvc/webcheck?revision=471&view=revision

Log:
pass a string to RobotFileParser because of problems with unicode

Modified:
   webcheck/webcheck/crawler.py

Modified: webcheck/webcheck/crawler.py
==============================================================================
--- webcheck/webcheck/crawler.py        Sat Nov 19 15:06:14 2011        (r470)
+++ webcheck/webcheck/crawler.py        Wed Aug 29 17:34:23 2012        (r471)
@@ -285,7 +285,7 @@
             # choose a link from the tocheck list
             link = tocheck.pop()
             link.is_internal = self._is_internal(link.url)
-            link.yanked = self._is_yanked(link.url)
+            link.yanked = self._is_yanked(str(link.url))
             # see if there are any more links to check
             if not tocheck:
                 tocheck = self.get_links_to_crawl(session)
-- 
To unsubscribe send an email to
webcheck-commits-unsubscribe@lists.arthurdejong.org or see
http://lists.arthurdejong.org/webcheck-commits/