Who it is
- Signature
Praha-X/0.1 (+https://praha-x.cz/bot)- Operated by
- Praha-X.cz, the city memory of Prague and the Czech regions
- Where it comes from
- our own addresses; the server is in the European Union
What it does
It reads publicly published news: RSS and Atom feeds of authorities and city districts, official notice boards, press releases of institutions and companies. From them we take the headline, the date, the link and a short context, so that we can write our own sentence about the event with a link to you as the source. We do not republish other people's texts.
For stories we write about, the crawler opens the article page the feed links to once and reads
its text as working material for our own summary. That text stays with us as working material and never
appears on the site. A text-and-data-mining reservation is respected (tdm-reservation, the
TDMRep protocol): where it is declared, we do not take the text.
For the Top 100 Czech websites ranking, the crawler opens the home page and the icon of about three hundred of the most visited Czech sites once a week: it checks whether the site is alive, where it redirects and what it is called. It reads nothing else from the site and does not store the content of the home page.
How it behaves
- It obeys
robots.txt: what is disallowed, we do not read. - It is slow: feeds several times a day, an article page only once, at most one request every three seconds per site, no parallel connections and no crawling of the whole site.
- It does not work around blocks. If you refuse us (with a 403, a 429 or a challenge page), we stop and record the source as closed. We do not try other addresses or other signatures.
- It does not pretend to be a browser and does not fill in forms.
- It collects no personal data beyond what is in the publicly published news item.
If you do not want us
One line in your robots.txt is enough: we will read it and leave.
User-agent: Praha-X
Disallow: /Or write to us through the contact form and we will remove the source by hand. We answer people, not robots.