Skip to content

Latest commit

 

History

History
87 lines (58 loc) · 3.48 KB

deindexing.md

File metadata and controls

87 lines (58 loc) · 3.48 KB

How to deindex/unlist/unpublish/block/remove your server's rooms from index

If your server publishes room directory over federation and has public rooms within the directory, they will appear in the search index after the next full reindex process (should be run daily).

Please keep in mind that to have a room indexed, you have to:

  1. Explicitly mark a room federatable when creating it
  2. Explicitly mark a room as public in room settings
  3. Explicitly publish a room in the room directory
  4. Explicitly publish your room catalog over federation

When you do all of those steps, you clearly understand the consequences of your decisions, i.e. a particular room that was made public, federatable, published in room catalog and then shared room catalog over federation will be accessed over federation.

MRS can't index a room if you didn't explicitly allow a room to be visible over federation. You either publish something over federation, or not. MRS is not a special thing, it uses the same API and the same set of rules as any other matrix server does.

unpublish your room directory from the federation

If your server is indexed, that means you explicitly published your public rooms directory over federation. If you don't like that the information you explicitly published over federation is accessed over federation, you should consider unpublishing it.

For synapse, you need to add the following config options in the homeserver.yaml:

allow_public_rooms_over_federation: false

in case of etke.cc/ansible and matrix-docker-ansible-deploy, add the following to your vars.yml configuration file:

matrix_synapse_allow_public_rooms_over_federation: false

However, if you think that "publishing over federation, but not for that particular member of the federation" segregation is a good thing for Matrix protocol, MRS has several options to unlist/unpublish/block/remove your server and its rooms from indexing.

(specific room) using room topic

As a room administrator/moderator, you could add special string to the room topic to prevent indexing:

<matrix.server_name from MRS config.yml>: noindex

Example:

example.com: noindex

(deprecated) robots.txt

While robots.txt is not part of the Matrix protocol, that's the easiest and convenient way to prevent your server from indexing. MRS supports several made-up endpoints to prevent indexing your server.

NOTE: due to the fact robots.txt is not part of the Matrix protocol, MRS treats it a bit differently than usual bots. User-agent: MRSBot MUST be present in your robots.txt file to make it work (User-agent: * will be ignored!)

whole server

Just add the following to your robots.txt:

User-agent: MRSBot
Disallow: /_matrix/federation/v1/publicRooms

specific rooms

Just add the following to your robots.txt (you can add multiple Disallow lines, one per room ID):

User-agent: MRSBot
Disallow: /_matrix/federation/v1/publicRooms/!yourRoomID:example.com

contact instance maintainers

MRS is open source project (code), it doesn't parse/index/process any data by itself, so you have to contact specific instance's maintainers. Each instance should have a page with details and contacts.

That method is discouraged, because it requires manual intervention by instance's maintainers, and thus each request processing will take way more time than using any of the methods described above.