Our crawler
This page explains who we are, what our crawler does, and how to ask us to stop visiting your forum. It is the address announced in our User-Agent.
What the crawler does
- Forums watched : over 4,000
- Sources listened to : over 19,000
User-Agent: FindMentionBot/1.0
How it behaves
- No. We read the RSS and JSON feeds forums publish to be read by machines, we respect their robots.txt, and we never open two simultaneous requests to the same site. A forum sees two to seven visits a day from us.
- We copy neither pages nor messages: only a title, a link back to the source, and the beginning of the message.
Getting removed
Write to the address below with your forum's domain. We remove it the same day, without argument and without asking why. You can also simply disallow us in your robots.txt: we read it once a day and we respect it.