IRC, Matrix, and thanks for all the kicks

Community
3

Thank you for the detailed summary @thibaultamartin !

Unfortunately, I recently had the problem again of being kicked out of several bridge rooms (e.g. Design / Hackers). I know this error message too well by now. This behaviour isn’t really newcomers friendly :confused:

I think we should enable that option, as it seems to be a good compromise.

In my opinion we should adopt a transition plan, and in the long run switch to Matrix, as also suggested by @thibaultamartin. Currently, Matrix is getting unfair criticism from some people, but this is often not a problem with Matrix itself, but due to the use of the IRC<->Matrix brigde. The bridge is a great help for the transition phase, but should no longer be needed in the end.

8 Likes
4

I have already explained my preference for Matrix in the survey. So just a big thank you to @thibaultamartin and all the people that make it possible that the tooling gets “out of the way” as much as possible!

6 Likes
5

After reading about the usability problems explained here, I have two suggestions that might be helpful.

  1. Last time I checked the wiki (both GNOME and Matrix), the procedure to store the IRC credentials in the bot wasn’t explained or was too hidden in the docs instead of being something that appeared upfront. This useful to avoid having to re-login and re-enter all +r rooms after each bridge reboot. In this mode of operation the bot uses the registered nickname you provide it as the puppet user on the IRC side from the first login attempt since it starts. I’m not familiar with the IRC minutae but my guess is that those protection measures against flood attacks apply for unregistered nicks. If that’s the case (or if it can be set up that way), that might solve most of the problems short-term (provided you have a registered nickname on the IRC side) without these split-brain issues you mention.
  2. The best solution longer term is to run only the Matrix server and proxy IRC clients with something like this. It provides the most seamless integration with no compromises since we get to keep only a single source of truth. The downside is that this path involves developing over that project or starting another from scratch. The upside is that it’s very likely that other communities have similar transition needs as us and/or it can make transitions possible where it was deemed too involved.

If we commit to the last option (which I argue it’s a noble one to pursue for the net effect and unpredictable but positive impact it can have on us and our place and reach in the FLOSS community), I strongly recommend joining developer manpower from us and other communities and work over the project I just linked (I expect they will welcome the improvements). Note that none of the options are mutually exclusive.

Edit: Nevermind the first option, it is already well placed here. Definitely things have changed since last time I had to deal with all this stuff when I was a rookie.

1 Like
6

That’s a very valid point. We have a more general onboarding problem. It’s not very clear for newcomers which services we provide, which ones are connected to a GNOME account, what a GNOME account is in the first place or even how to get one. I need to make room in my schedule to finish our service catalogue, and what each user profiles (anonymous, registered user, GNOME Foundation member) can get you.

Since refactoring our onboarding process is a tedious task we need to address in the long run, I had updated the procedure to register a Matrix puppet against IRC is now part of the IRC/Matrix onboarding guide.

It would indeed be the cleanest way to do it because at least a single system would be authoritative on the accounts. As you said, it would even be a single base of accounts and it would “just” be proxified.

But people using this interface would still connect through the IRC protocol. Some limitations of IRC can’t be overcome. People using this proxy wouldn’t support edits, replies, and code formating. Those the main complaints IRC users have today when it comes to interactions with Matrix, and such a proxy couldn’t address them.

Deploying a non-trivial amount of paid and volunteer efforts to only address a tiny bit of the problem (the NickServ spells) is probably not worth it.

The MR Element has landed on the current bridge has a way better effort/benefit ratio. In practice it means every other month people from IRC might miss some messages during a few minutes. That’s quite an acceptable tradeoff if you consider it as a legacy system :slight_smile:

7

That’s a limitation of the IRC protocol, as you say. If IRC users complain about the lack of features of the service they use, well, I guess the solution is on their hands, not ours. The feature mismatch can be made nicer with some human-alike-but-trivial-enough translation rules tho (like using an external service for code blocks that exceed some length and sending a link through IRC), but that can be done both in the bridge and the client proxy.

If you aim to solve only that problem it looks like it is overkill. I don’t say “it is overkill” because some people might not be happy with this inconvenience. Legacy doesn’t mean unreliable where it wasn’t from a UX perspective. I’m personally OK with this tradeoff, but we have to consider other scenarios.

However, an enough-featured client proxy would change things at a more fundamental level, to the extent of making the experience of creating and managing rooms the same as full native Matrix. No complications or having to think about the bridge when doing those tasks. Then there’s the argument of thinking at the whole FLOSS world level and actively making IRC a thing of the past and all that it means for its future and newcomers everywhere by providing such strong means for it without leaving IRC users behind and forcing them to another UX, but that is a whole bigger topic, almost philosophical, and goes beyond us.

8

On the “Libera Chat” spam

First, of course, we know that despite what the spam indicated the spam did not actually come from Libera Chat teams. It comes from imbeciles who obviously wanted to give Libera Chat a bad image by flooding all sorts of disgusting messages.

What happened?

Let’s have a clear look at what actually happened.

  • Many IRC channels were flooded with spam messages. Given our Matrix instance and the IRC network we rely on are bridged, the flood messages were forwarded to Matrix
  • Those IRC channels were not in +R mode. This is the mode that only allows people who are registered on IRC and have cast the NickServ spells to join
  • Most of those channels were not set on +R mode during the flood
  • Since the flood matched simple expressions, it would have been easy to prevent it at server level but it didn’t happen
  • Several people have been granted privileges on Matrix and have been running scripts to clean up all the garbage left by IRC spam
  • New versions of our Matrix instance have been deployed an hour after the spam started, with more agressive spam filtering so the messages didn’t make it through anymore. Spam kept happening on IRC but didn’t make it through Matrix.

What can we learn?

As one of the persons who had to manage all the cleaning of the spam on the Matrix side, this was a very frustrating evening. It really felt like bailing out water of a boat with huge holes in the hull. And to be completely honnest: this was not a very sophisticated nor massive attack, making the struggle even more infuriating.

The easy solution could be to blame the computer. I could ragefully blame IRC, and actually did during the flood, out of exasperation. But the problem is not IRC. The problem boils down to two major issues:

  • The registration/authentication experience on IRC is really cumbersome: as a result, most IRC channels don’t have a lock on the door
  • We don’t have a proper moderation team with sufficient privileges to react quickly on IRC, and worse: we don’t have sovereignty over the IRC network we use to be able to set-up spam filters at server level. People who run it are volunteers who do their best on their spare time, and there is no real decision-making process nor way to enforce decisions on all servers of the network. It leaves the door open for spam on both platforms and is an effective risk for our users

The key takeaway after discussing with the GIMPnet administrators is that GIMPnet is running in maintenance mode. The administrators are keeping the network running, but don’t have the time or energy to carry out significant evolutions.

What is our way out?

The first solution that comes to mind would be why don’t you just put a lock on the doors then? And of course, like many of the why don’t you just questions the answer is not as simple as it seems. Today, GIMPnet is not making any exception for the IRC/Matrix bridge. Setting blindly all channels in +R mode on IRC would degrade the experience both on IRC and Matrix:

  • IRC users would need to cast the NickServ registration and and authentication spells
  • Matrix users who didn’t register on IRC and taught the bridge to play their credentials could just not join the conversation
  • It would require every existing and new channels ops to set-up the +R mode

Not great.

But I still have pretty high hopes that we can do better together. I had a very short and informal conversation with GIMPnet administrators who told me:

this recent upheaval has pointed out that there might be some need for a bit more modern processes and maybe getting some more active staff in that might have new and bright ideas

We still need to have a more formal conversation with the GIMPnet administrators and @averi to push the idea forward. If we can manage to have special privileges for the IRC/Matrix bridge on GIMPnet, we can find solutions to automate spam management without degrading the experience.

It is possible to set more agressive anti-spam settings on IRC, all while putting the bridge on an allow-list. This would require a bit of tweaking on the IRC side, but would definitely make the experience better at both ends.

9 Likes
9

As part of Fedora’s transition to Matrix, we’ve plumbed through all the rooms and we configured the rooms so that Matrix users are exempted from the registered nick requirement through the access list:

Excerpted from the #fedora-kde access list:

[20:11:51] ChanServ [ChanServ@services.]: Entry Nickname/Host          Flags
[20:11:51] ChanServ [ChanServ@services.]: ----- ---------------------- -----
[20:11:51] ChanServ [ChanServ@services.]: 8     *!*@gateway/shell/matrix.org/* +V [modified 29w 4d 2h ago]

Perhaps something similar could be done for GNOME as a transition measure?

2 Likes
10

It’s always nice to see what sister projects are doing, thanks for letting us know @Conan_Kudo!

Unfortunately, GIMPnet is powered by unrealIRCd which doesn’t allow such exceptions. I’ll bring the issue to GIMPnet administrators anyway to make sure I didn’t miss something in the docs :slight_smile:

11

Probably not a surprise to anyone, but I’d also advocate for fully switching our communications to Matrix in the long run.

It also makes sense to me to enable the option and prevent kicking Matrix users from bridged channels, those random kicks are the last major annoyance with Matrix in my opinion.

1 Like
12

Thanks for this excellent summary.

In case folk aren’t aware, I would like to highlight another solution / workaround for chat fragmentation. Most chat protocols are now accessible through the Web, and some apps provide specialized browser UIs suitable for managing multiple chat webapps. The two I know of are Tangram (GTK-based) and Ferdi (Electron-based).

To access IRC from a browser I use TheLounge web client, there are probably others too.

I would still love to see GNOME nominate an official platform, for reasons of community cohesion more than anything :slight_smile:

1 Like
13

@av and I had a really uplifting conversation with (some of) the GIMPnet administrators!

The key takeaways from the conversation are:

  • IRC opers are depleted and would happily welcome new staff to help fight spam
  • the IRC network will undergo routine maintenance for updates shortly
  • once everything is up to date, the admins are open to update the config to:
    • whitelisting the bridge so it doesn’t trigger flood protection (what cause the kicks right now)
    • exempting the bridge puppets (Matrix users created on IRC by the bridge) from having to be registered on GNOME channels when IRC channels are in +R mode

Really uplifting news :tada:

6 Likes
14

Thank you @thibaultamartin for the excellent write-up and the glimpse into our messenger history.

Very much agree. Thanks much for the excellent and well written write-up @thibaultamartin :slight_smile:

15

It was a pleasure to read and really gave me new interesting insights into the history of Instant messaging.

16

Speaking of it: I wonder how the “Chat Platforms Evaluation” is going :slight_smile:
@nmcgovern

2 Likes
17

Thank you for the detailed explanation of the problem space @thibaultamartin. I know how much work this represents :slight_smile:

I believe what is missing here now for people like me waiting for “dust to settle” before migrating would be a “plan & schedule” for the migration (if applicable) of GNOME’s Matrix instance and accounts/data from Element’s servers to GNOME’s. I would like to know when/where/how I would have to register an account to be all set for the next 100 years, without having to do it more than once…

Also, ignoring the whole IRC question for a minute: what would spammer control mechanisms look like in a hypothetical “pure Matrix” world?

18

Thanks for allowing me to introduce what’s left to do Jeff :grinning_face_with_smiling_eyes:

That’s a very fair question, which requires a solid answer to be acceptable. The initial post of this thread is the first part of a series. I also blogged something about how to explain Matrix with e-mails, to make it easier to understand for people less familiar with the subject. In its conclusion I say

A post on how an organisation can keep its sovereignty in a decentralised and open network as Matrix will follow in the coming weeks.

That’s another very important aspect: Matrix is an open network, but it doesn’t mean we can do anything we want and expect it to work like a charm. The migration from Element’s to GNOME’s infrastructure is one of the steps of a broader plan. We need to think it through first, and to set-up a set of rules to enforce our strategy. My mind is pretty clear on that point, but rather than throwing ideas as I’m sipping my first coffee of the morning, I’d rather wait for a detailed blog post to explain it all in depth.

Then comes the last part, the one you mention: we have a Matrix instance in a given state, we will have a target, we need a transition plan to reach that target. By a happy coincidence, @averi is re-working our identities management system. Most of the steps on the Matrix side are clear to me, but instead of throwing ideas in a mess I would rather wait for our identities management system to be advanced enough to finish up and formalise the transition plan.

Matrix has pretty solid, battle-tested defence mechanisms now. The Matrix.org instance has more than 100k Monthly Active Users (MAU) and of course has triggered spammers’ interest. The most immediate defence mechanism is against individual troublemakers: a moderator in a room can kick/ban a user from a room, and redact their messages so they are removed from the timeline.

More sophisticated attacks can happen, for example when a spammer has automated the spamming in several rooms. That’s where the hammer of Matrix justice comes to play. To keep things short, it creates a room in which a moderation team can send commands to a bot to ask it to kick/ban a user and redact their messages for all the rooms under its protection. Just a note: it doesn’t make sense in the Matrix universe to talk about “rooms belonging to a server”. What makes more sense is to talk about “rooms we control”: that’s why the moderation by mjolnir is opt-in. When people create a room and want to be protected by mjolnir, they need to grant it the administrator privileges to be placed under its protection.

Finally, for the more extreme cases there are two last resort mechanisms: putting an obviously maliciously crafted instance on a denylist, and for the most extreme cases shadow banning them.

That’s where things get really good: with the proper authorizations on the IRC side it is possible to kick and ban users from the Matrix side! For more sophisticated defence such as changing the modes of a channel, manual work has to be done on IRC. We’re still working with GIMPnet administrators to upgrade the network, and we’ll be able to retrieve proper rights to do it.

19

I see you’re not on Planet GNOME, and not DuckDuckGo-able either. Where is your blog? (You probably want to be on Planet GNOME!)

1 Like
20

The article I mention is Matrix for Instant Messaging :slight_smile:

I need to take the time to figure out tags with Hugo and I will open a MR to be added to Planet :grinning_face_with_smiling_eyes:

3 Likes
21

About the IRC update, @averi has been following progress with the network administrators.

  • We are waiting for some administrators to confirm they have upgraded the hosts on which ircd is running and are ready to move forward upgrading unrealircd. We expect the update to land in the next two weeks, but it depends mostly on the schedule of the IRC administrators (who are providing the service on their free time)
  • Once done, we can land the flood prevention whitelist for Matrix (to prevent bridge reboot kicks) and the rule to exempt Matrix puppets from needing +R mode in +R IRC channels (to prevent kicking/join deny of Matrix users when a IRC channel is in spam-protect mode)
  • We may eventually add a cloack for the IPv6 subnet of Matrix to identify more obviously Matrix users on IRC
1 Like
22

I just came here after following a link from the Fedora Discourse. I’m not too active on GNOME channels, so my opinion might not mean much. I’m writing my thoughts here because I think this discussion is relevant for many other communities, and this decision could help cement an existing precedent far beyond GNOME itself.

I think maintaining a bridge is currently the “least bad” option available. I’ve written my thoughts on this in an email on the Libravatar-Fans mailing list. I’ve also described some redeemable issues I have with Matrix in a section of a blog post, linked below.

Pasting my email below:

Personally, I don’t have a big problem with either since both are open platforms with some degree of federation; I use both daily through WeeChat and Gomuks. However, I do have a preference for IRC for a number of reasons:

  1. Given that so many other projects are on IRC, it makes sense to not require people to use a different client just for a few select communities.

  2. Many people, including myself, prefer TUI clients to graphical ones. Right now, the only TUI Matrix client that isn’t missing essential features is Gomuks. While Gomuks development seems to be progressing well, I wouldn’t say it’s a replacement for other graphical clients yet.

  3. Issues with Matrix itself: I described these in a bit more detail in a blog post, “Keeping Platforms Open”. https://seirdy.one/2021/02/23/keeping-platforms-open.html.

Regarding features in Matrix that aren’t present in IRC:

Long-form and long-term discussion already happens in a mailing list, which is well-suited for the task; I’m not aware of any other open platform that allows nested discussion threads delimited by subject. Given the existence of a mailing list, I think a chat platform should focus on a niche that isn’t covered by mailing lists: ephemeral, real-time chat with less structure.

If a discussion needs marked replies and searchable history, it’s probably better off happening in a mailing list. Given that these features aren’t especially valuable given the existence of a mailing list, I’d say that IRC should fit the bill.

Honestly, I don’t think a Matrix-IRC bridge hurts the Matrix experience too much since join/leave events can simply be filtered out from most clients. Most of the issues come from the perspective of IRC users, mainly long-form messages turning into pastebin links instead of being broken up. That being said, I find excessive pastebin links preferable to not having IRC support at all.

Not all of what I said in the Libravatar-Fans mailing list is applicable to every GNOME project. For example, Libravatar maintains both a mailing list and an IRC channel; this is not true for all GNOME projects.

I think that the issues involving NickServ registration are valid. Perhaps efforts could be spent on a bot that kicked unregistered IRC users but not users connecting from Matrix? I’ll see if I can give this a shot next week if there’s enough interest. Nonetheless, while I agree that the registration issues are bad, I don’t think they’re worse than abandoning IRC.

I’m currently in the process of expanding the thoughts I described in that email into a {Web,Gem}log post, so any feedback before then is welcome.