[gtld-tech] [weirds] Search Engines Indexing RDAP Server Content

[James Galvin]

This is not a very good example of the need for operational practices.

It represents Google UK’s view of what you can find when you start at 
“http://rdg.afilias.info/rdap/help”, which includes explicit 
examples of how to use the RDAP server.

Jim



On 29 Jan 2016, at 8:34, Hollenbeck, Scott wrote:

> (cross-posting to multiple lists - sorry if that's inconvenient)
>
> So I saw a tweet from Gavin Brown (@GavinBrown) that describes how one 
> particular search engine has indexed the RDAP server of a gTLD 
> registry operator:
>
> https://twitter.com/GavinBrown/status/692718904058191872
>
> This is all the more reason to work on a client authentication 
> specification that includes support for varying responses based on 
> client identity and authorization. I've been working on such a 
> specification and welcome feedback on the approach:
>
> https://datatracker.ietf.org/doc/draft-hollenbeck-weirds-rdap-openid/
>
> It also begs the question of the need for a BCP describing operational 
> practices for server operators. There are ways for web servers to 
> influence or restrict crawler behavior, but what's appropriate in this 
> context?
>
> Scott
>
> _______________________________________________
> weirds mailing list
> weirds at ietf.org
> https://www.ietf.org/mailman/listinfo/weirds