Build crawler humanly as different roles which be combined with different components.
pip install smoothcrawler