[UA-discuss] Web Crawlers behavior on UA

[Dessalegn Yehuala]

Hi All,

The robot.txt file is encoded in UTF-8 format and I am just curious if
there has been any study or analysis done on  web crawlers behavior on UA .


Kind regards,

Dessalegn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mm.icann.org/pipermail/ua-discuss/attachments/20200108/39298141/attachment.html>