pepy.tech

protego

0.6.0
105.42M

Pure-Python robots.txt parser with support for modern conventions