Running your own Matrix server and use existing identity servers

Hi all,

not sure if this is the best place to ask, but I think some people here have experience in running their own Matrix server. So let’s give it a try.

I tried to setup my own Matrix server and it seems that it work more or less. What I noticed is that when I go to my profile settings and enter my email address, I get this error:

Adding an email to your account is disabled on this server (Status 400)

Same if I try to enter my phone number. Being searchable by other people with public ID’s like my email addresses or my phone number is something i really like to have.

I assume that it is possible to connect my Matrix server to one of the big identity servers (matrix.org or vector.im) to publish my users including their email address and phone number, if they wish. But I couldn’t find the corresponding config switches.

I already set:

default_identity_server: https://matrix.org
trusted_third_party_id_servers:
   - matrix.org
   - vector.im
enable_3pid_lookup: true
allowed_local_3pids:
    - medium: email
      pattern: '.*'
   - medium: msisdn
     pattern: '\+49

but this doesn’t seems to be enough…

Any hints?

1 Like

I’m one step further. For email verification I had to configure the email notification settings first… makes sense :see_no_evil:

But I still struggle with the phone number thing… But therefore I probably need a service run by myself which is able to send out the verification SMS, right? I probably can’t use the one from vector.im and/or matrix.org. How does others handle it? Is there a easy to use public service available? How do we do it on the feneas Matrix server for example?

TL;DR: set following config parameter to get your SMS working again

account_threepid_delegates:
    msisdn: https://vector.im

The current documentation on this is very thin. I think its because they are still unsure how to properly implement this into synapse.

Have a look at https://github.com/matrix-org/synapse/blob/master/UPGRADE.rst and go to the " Upgrading to v1.4.0" section.

As of this release, users will be unable to add phone numbers or email addresses to their accounts, without changes to the Synapse configuration.

further on

Delegated email verification will be disabled on Monday 2nd December 2019 (giving roughly 2 months notice). Disabling delegated SMS verification will follow some time after that once SMS verification support lands in Synapse

so now you have to setup a own sms gateway and delegate the job via the configuration variable account_threepid_delegates (see https://github.com/matrix-org/synapse/blob/master/docs/sample_config.yaml#L1021) or use vector.im for the time being even though it will be obsolete in future.

Feneas is still using the vector identity server for SMS verification but it is not required on registration. As soon as they announce an EOL date or implement sending/using SMS gateways within synapse we should think about doing something on our own. This could be also a great topic for FOSDEM 2020 (@jaywink) :slight_smile:

2 Likes

Thanks @zauberstuhl SMS and mail verification works now :slight_smile:

Today I tried to join feneas:feneas.org. The user list gets loaded but fetching the messages fails with this error:

2019-12-23 12:40:43,355 - synapse.http.site - 203 - WARNING - GET-28- Error processing request <XForwardedForRequest at 0x7f3306a71160 method='GET' uri='/_matrix/client/r0/sync?filter=0&timeout=30000&since=s1636_1016_0_9_3_3_1_16_1' clientproto='HTTP/1.1' site=8008>: <class 'twisted.internet.error.ConnectionLost'> Connection to the other side was lost in a non-clean fashion: Connection lost.
2019-12-23 12:40:46,155 - root - 237 - WARNING - None- ***** STARTING SERVER *****
2019-12-23 12:40:46,155 - root - 238 - WARNING - None- Server /opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/app/homeserver.py version 1.7.2
2019-12-23 12:40:49,384 - synapse.http.matrixfederationclient - 522 - WARNING - GET-8- {GET-O-2} [t2bot.io] Request failed: GET matrix://t2bot.io/_matrix/federation/v1/backfill/%21XTcFGXYoSwwJPindtt%3Afeneas.org?v=%241577062562505aahiU%3Afeneas.org&limit=100: HttpResponseException("403: b'Forbidden'")
2019-12-23 12:40:49,387 - synapse.http.server - 109 - ERROR - GET-8- Failed handle request via 'RoomMessageListRestServlet': <XForwardedForRequest at 0x7fdc565cf320 method='GET' uri='/_matrix/client/r0/rooms/!XTcFGXYoSwwJPindtt%3Afeneas.org/messages?from=s1636_1009_0_9_5_3_1_16_1&limit=20&dir=b&filter=%7B%22lazy_load_members%22%3Atrue%7D' clientproto='HTTP/1.1' site=8008>
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/http/server.py", line 77, in wrapped_request_handler
    await h(self, request)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/http/server.py", line 326, in _async_render
    callback_return = await callback_return
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/rest/client/v1/room.py", line 517, in on_GET
    event_filter=event_filter,
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/handlers/pagination.py", line 350, in get_messages
    room_id, max_topo
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/handlers/federation.py", line 1016, in maybe_backfill
    success = yield try_backfill(likely_domains)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/handlers/federation.py", line 986, in try_backfill
    dom, room_id, limit=100, extremities=extremities
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/handlers/federation.py", line 759, in backfill
    dest, room_id, limit=limit, extremities=extremities
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/federation/federation_client.py", line 194, in backfill
    dest, room_id, extremities, limit
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/http/matrixfederationclient.py", line 776, in get_json
    timeout=timeout,
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/http/matrixfederationclient.py", line 246, in _send_request_with_optional_trailing_slash
    response = yield self._send_request(request, **send_request_args)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/http/matrixfederationclient.py", line 477, in _send_request
    raise e
synapse.api.errors.HttpResponseException: 403: b'Forbidden'
2019-12-23 12:42:56,591 - synapse.http.matrixfederationclient - 487 - WARNING - federation_transaction_transmission_loop-12- {PUT-O-55} [matrix.koehn.com] Request failed: PUT matrix://matrix.koehn.com/_matrix/federation/v1/send/1577101246278: DNSLookupError('no results for hostname lookup: matrix.koehn.com')
2019-12-23 12:42:56,597 - synapse.http.matrixfederationclient - 487 - WARNING - federation_transaction_transmission_loop-11- {PUT-O-61} [linuxgaming.life] Request failed: PUT matrix://linuxgaming.life/_matrix/federation/v1/send/1577101246284: DNSLookupError('no results for hostname lookup: linuxgaming.life')
2019-12-23 12:42:57,178 - synapse.http.matrixfederationclient - 487 - WARNING - federation_transaction_transmission_loop-26- {PUT-O-13} [swedneck.xyz] Request failed: PUT matrix://swedneck.xyz/_matrix/federation/v1/send/1577101246236: ConnectionRefusedError('Connection refused')
2019-12-23 12:42:57,186 - synapse.federation.sender.transaction_manager - 154 - WARNING - federation_transaction_transmission_loop-6- TX [permaweb.io] {1577101246228} Remote returned error for $15771013760MtAYC:schiessle.org: {'error': "ERROR 403: Your server isn't divulging details about prev_events referenced in this event."}
....

It seems like I can’t connect to the Telegram bot but also to other homeservers in the channel…

I can join other federated rooms like matrix-news:matrix.org so I don’t think it is a general federation issue on my side…

Any idea what could be wrong here?

After waiting long enough the room history was loaded/synced. Still the error message doesn’t make me feel comfortable. Especially as the messages continue to appear in the log since I joined the federated room.

The error looks like it’s failing to fetch some certain messages from t2bot server regarding this room in the backfill. You’re likely to see this error disappear once the message becomes so old that it wont be tried any more, possibly? Difficult to say why it’s failing. Possibly some key issue or out of sync room state. Might be worth looking at issues in the Synapse issue tracker to see if something familiar comes up.

Note that our Feneas room is still version 1 of the spec, which is likely the reason for state mismatch issues if any. We should really upgrade it to the latest recommended v5, but need to do it with time to not break the bridges. We’ll probably do it in our hackathon pre-FOSDEM in Brussels.