Robots
The Robots module provides the following functionalities:
- contains a router to match application action class for requests to the
robots.txt
file; - allows obtaining the content of the
robots.txt
file depending on the settings of the current website.