Hey all, it's time for another Synapse release! That's right, Synapse
1.63 is out,
let's have a look at it.
Clarifications on "anonymised" server statistics
Synapse has the ability to report usage statistics to the Matrix.org Foundation
(or to another location, if configured to do so). These statistics, such as
number of users, number of rooms joined by the server, etc. (they don't feature
any identifiable information about users and rooms) help us monitor the health
of the public federation.
In this release of Synapse, we have updated our public documentation about this
feature to clarify how it works and what exactly is being reported. This
documentation can be found right
here.
Note that previous documentation and prompts surrounding this feature called it
"anonymised" server statistics. This could easily be misinterpreted, as while
per-user statistics are not reported, homeserver server names are. We have
therefore changed said documentation and prompts to be clearer about
what is actually reported.
Note that your homeserver will never report any statistics if the report_stats
configuration option is set to false. Server admins who are curious about
which software is used by the Matrix.org Foundation to record server statistics
can check out panopticon.
Improved filtering for public room search
This version of Synapse ships with an experimental implementation of
MSC3827 which
allows filtering public room search results by room type. This feature will
enable better discoverability for Matrix Spaces (which are rooms with a specific
type, under the hood), as it will enable clients to search specifically for
public spaces.
This feature is still experimental as its
MSC hasn't
completed the MSC process yet, though it is in its final comment period at the
time this post is being written. This means a stable implementation will be
coming to Synapse very soon, so watch this space!
Everything else
Synapse 1.63 also includes a new rate limiter to limit invites per issuer. This
rate limiter can be configured using the new rc_invites.per_issuer
configuration setting, see the
documentation
for more information.
See the full
changelog for a
complete list of changes in this release.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper,
villepeh and Petr
Vaněk, as well as anyone helping us make Synapse
better by sharing their feedback and reporting issues.
This week Matthew gives us a general update on everything Matrix and Element. What is going on with Sliding Sync, OpenID Connect or P2P Matrix? Let's find out!
We're also pleased to announce the next episode of Open Tech Will Save Us has been scheduled. We will be talking live and interacting with you on July 27, 16:00 UTC (18:00 CEST) on the subject of platforms with people from GNOME, KDE and Element.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
The Spec Core Team took a concentrated effort to work through our priority backlog of MSCs, resulting in the decent amount of MSCs hitting FCP this week!
Thankfully the community has nearly matched this number by adding quite a few more MSCs into the fray. But we're still up by 1!
Perhaps one of the most exciting MSCs to move forwards is MSC2676: Message editing, which allows users to edit events (including messages) that they've previously sent. Part of the aggregations work, this functionality has been used extensively throughout the ecosystem. But only now is it making its way into the spec proper. I'm also personally excited about MSC2285: Private read receipts, which allows users to read a room's contents without advertising it.
Of course the above MSCs are still in Final Comment Period; a 5 day period where anyone can raise their concerns about an MSC before it is accepted. So if you have any last-minute comments, be sure to get them in now!
Another MSC I'm personally excited about! This MSC provides an endpoint on the homeserver to request the rooms (or spaces) you have in common with another user. I would find this useful for incoming DMs from users I may or may not know. Note that the client may not already have this information if it is making use of lazy loading room members, so it needs a way to ask the server for this information.
Looks like the MSC is currently sitting in the Spec Core Team's review backlog. We'll get to it soon!
The Synapse team has been hard at work! This week we published Synapse 1.63.0rc1. Among the notable features are support for MSC3827: filtering /publicRooms by room type, allowing for better discovery of Spaces. You can read more about that here. In addition, the release contains a number of bugfixes, updates to the documentation, and internal changes focused on reducing memory usage and increasing performance, as well as supporting the long-standing goal of faster room joins. More on these next week in the official release announcement!
Good news, as you might have noticed, conduit.rs is back online and better than ever! Matrix.org is so kind and donates this Linux server to me. I already host a discord appservice and handle 23k messages per day and, most importantly, now I'm at the top of both #ping rooms ;)
On the development side, I'm currently working on a big refactor, see https://gitlab.com/famedly/conduit/-/merge_requests/365 and please give feedback if you have experience this is your chance to have a lasting impact.
rofi is a application launcher and dmenu replacement. https://github.com/davatorium/rofi. I made a little plugin for it to switch channels on the nheko matrix client https://mzte.de/git/LordMZTE/rofi-nheko
You can now edit aliases in Nheko! This means you can publish your own aliases in the room directory as well as in the room. If an alias can't be used by people (because it isn't listed in the room directory), that alias is highlighted in red with an easy option to fix it (if you have permissions to do so). You can also easily switch the primary (canonical) alias of the room.
With this the feature set I wanted to have for the next release is complete. I wanted to focus on improving moderation and room management capabilities in Nheko and it does now have a very basic sets of capabilities to do so. I do already have plans to expand on them though!
So instead of working on new features, I will be focusing on bug fixes. For a start if someone has the username room, Nheko will now omit the reply fallback to prevent pinging the whole room. The verification window now also should always be big enough to show its contents and we now properly explain to users, why emojis might look different on different devices fixing an issue reported via Twitter... There is also a new icon for the room directory (it is now a building, very punny, huh?).
Tobias has done some internal refactoring, which will allow us to create more automated tests and prepare for the eventual Qt6 migration
Nvrwhere has improved NeoChat's Timeline layout: On a wide window, the bubbles will now be centered in the window, in order to better use the space while not stretching the timeline out too far
He's also fixed several papercut issues around the UI, for example editing and replying using the keyboard shortcuts
The browser will now properly open when clicking on a link when using wayland
The first part of Snehit's GSoC project is almost ready to land: The room list is gaining a list of spaces, which can be used to filter for the rooms included in that space
We're also moving forward on the E2EE support, with NeoChat now allowing a user to send an encrypted message into a matrix room - provided libQuotient supports it 🙂
Tobias is also working on improved Reaction & Emoji pickers, which will allow users to select different variants like skin tones and which will behave better on mobile devices
After some successful testing on current registration flows, we are running testing sessions on the new and improved registration flows over the next week.
We are also moving away from spreadsheets over to TestRail, a test case management tool which will help us track regressions and issues. We’ll be offering our community testers a change to try it out over the next few weeks.
Started experimenting with matrix-react-sdk for an improved crypto(graphy) experience
The share screen button in video rooms on desktop has been temporarily removed until the underlying issues can be resolved
We have removed most of unused code in the Element Web repositories and are working on setting up tools to avoid forgetting to remove unused code in the future
The new search experience and DM flow have received some bug fixes and polish in addition to more polish to CSS
Final commenting period expected to start next week
We’ve been tackling our process around reviewing PRs to reduce review time, taking a two pronged approach at focusing on reviewing new PRs quickly and resolving our oldest non-draft PRs every week
Did you know that some of our issues in GitHub are labelled with “Help wanted” and “good first issue” if they’re especially suited for community contributions?
In the works (you can enable labs features in settings on develop.element.io or on Nightly):
Version 1.8.22 RC was released to TestFlight and comes with the following changes:
In-app notifications are now available.
A shiny new offline indicator that's visible in more parts of the app.
A handful of fixes for issues when making calls.
Unfortunately this build was rejected following the latest requirements for account deletion in the App Store Review Guidelines. We’re looking into ways we can resolve this problem.
We have started to use Sentry to provide more insights into technical issues encountered whilst using the app.
Work continues on implementing the new app layout.
1.4.28 is being prepared with the following changes:
Makes the build process compatible with F-Droid again!
Fixes for voice messages not playing and some characters showing in their escaped html form - such as quotes showing as "
Nightly builds are on the way, these builds will be under a separate Application ID allowing them to be installed alongside the production Element app. Expect more information to come soon.
We’re investigating general performance regressions across the app, with some improvements already in the pipeline.
In the last two weeks or so, we've made some UX improvements:
unified SSO and ordinary login cards
unified media and regular file-uploads, and improved drag-and-drop support
improved feedback while validating login and registration input
We also added one feature that friends of mine were requesting 😃: annotations can now be equipped with a "motivation" (part of the w3c web annotation data model, mentioned in MSC3574), and filtered by motivation. This makes it easy to, for example, mark some discussions as "questions" about the text, which is pretty useful when managing a class.
And I've now finally managed to cut a long-overdue new 2.6.0 release of the Ruby SDK, which includes better support for concurrent multi-threaded usage as well as improvements on Ruby 3.0+. (The tests are currently green for Ruby 2.6 to 3.1)
The team is proud to announce the first release of @matrix-org/matrix-sdk-crypto-nodejs to npmjs.com (0.1.0-beta.1) for all the best npm user friendliness you can get. This marks the first release of the crypto ffi for nodejs from the rust team supreseeding the previously existing project. It is also the very first release for nodejs featuring the great new vodozemac core (replacement of libolm). We've marked this as a beta release while we work out any remaining minor hickups or problems, so if you see something, say something! However, the main point of contact to it for most people is probably TravisR |s Nodejs Bot SDK, which is already consuming that beta in its latest release.
After a year of working on different ways to support encryption, the bot-sdk has finally stabilized on the rust-sdk crypto crate for crypto bindings internally. Tutorials on how to get your bot/appservice set up and working with the new crypto code are here (note that for appservices you'll need to turn on EDUs and MSC3202 in Synapse). If you run into problems, please report bugs or pop by #matrix-bot-sdk:t2bot.io on Matrix for help.
Alongside support for encryption, v0.6.0 brings a whole lot of other functionality for appservices, Synapse Admin API users, and bots wanting to make use of real DMs. Check out the full changelog here.
axon.sh v0.17.0 has been released which adds support for managing per-user ratelimits, and includes a fix for a bug related to server discovery. Please try it out and report your experience in #axon:matrix.thisisjoes.site, I'd appreciate feedback!
A whole heaping of work has landed for my release tracker project, with it now supporting tracking releases for projects (loose projects, all users stars, or all projects under a namespace) on both GitHub and GitLab (for .com as well as self-hosted instances).
Documentation is currently still a bit lacking, but hoping to have more information both on setting it up as well as using it Soon™.
my conference talk at ACCU this year is now out on YouTube: Matrix is a Distributed Real-time Database. I explain what Matrix is, and show how to use curl to send and receive messages, and why it's "interesting" to write a server, and, as the title hints, I introduce the idea that Matrix can be for a lot more than messaging.
Reminder: Matrix Summit Conference Berlin call for participation ends next week
As announced earlier, there will be a conference all about matrix. (See TWIM 2022-07-01)
We encourage everyone with matrix-related projects, products and ideas to come around and give a talk: From the past to the future, from the moment of the idea, the story of the creation or the vision of the future. We’d like to understand the principles as well as the technology. The conference is from people for people, so if you’d like to talk about yourself, your community, your organization, please do. Showcase yourself and your relation to the Matrix world. Let's get to know each other!
We aim to compile a versatile program. We are open to contributions of any length, from 5 minutes (lightning talks) to presentations and talks to workshops and hacksessions up to 5 hours. We’ll come together to discover, celebrate and enjoy the world of matrix. Also, if you have any arty, cultural or playful contribution in mind, please offer it.
➡️ Proposals for the Matrix Summit in Berlin can be submitted until next Friday (2022-07-22 22:22, TZ=Europe/Berlin) via pretalx. 📝
Rick and I talk about the the brand new Chatterbox. Is it Riot Embedded rebranded? Did the world need yet another embedded chat solution? Can I host mine? Can I make it clever? Let's find out!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
While the name may not make much sense to the layperson, the idea of a "canonical" Direct Messaging room (DM) is one that would always be referenced whenever a DM between two people is requested. That is, instead of potentially having a few different DM rooms with someone, both you and the other person would always know which room to use when DM'ing each other.
We don't really have this today. DM rooms are just group rooms with only you and someone else in them. If you attempt to DM another user, your client will try to guess the best room to use for this through some clever heuristics. What's lacking is a defined way to always arrive at the same room for this action.
This MSC attempts to define one, and would allow other functionality to be built on top of it, such as definitively knowing which room to send user-to-user data into, and to read from.
This week the Synapse team released Synapse 1.62! It features a lot of changes, including a fairly big update of spam checker callbacks, performance improvements around syncing and device management, improved customisation of .well-known client files, and much more. Read all about it on the Matrix.org blog: https://matrix.org/blog/2022/07/06/synapse-1-62-released
Apart from this, we've been working on refining and fine-tuning our processes as a team over the past few weeks, which, among other things, resulted in the creation of this documentation that gives contributors some insight on how we review pull requests on Synapse. Olivier has also landed his work on running Complement (our next-gen integration test suite) against instances of Synapse using workers, which is a massive improvement for our CI.
Hello again TWIM, updates to my Kubernetes charts have been rolling along as usual, though I've been a bit more silent about them. This week sees the addition of a synatainer chart though, for those who want it for maintaining their K8s Synapse. (And the Synapse chart was also updated to 1.62.0, and element-web to 1.11.0)
If you have any questions, comments, requests for assistance, etc with them then #matrix-on-kubernetes:fiksel.info is where you want to go.
The k8s-at-home dendrite helm chart now optionally configures ingress resources for dendrite in polylith mode. It has also had some bugs fixed and been updated to support the most recent version of dendrite.
Check out the chart here: https://github.com/k8s-at-home/charts/blob/master/charts/incubator/dendrite/README.md
Quadrix v1.2.0 has now been released. It's already available for Linux (https://snapcraft.io/quadrix, https://flathub.org/apps/details/chat.quadrix.Quadrix) and Android (https://play.google.com/store/apps/details?id=chat.quadrix.android). The Windows, MacOS and iOS versions are awaiting approval from the respective stores.
New in this release:
Brand new icons from Remix Icons (https://github.com/Remix-Design/RemixIcon)
Messages can be redacted (for now only by the message owner)
Users with admin power can kick other users from rooms
Users can start a DM room directly from the member list in a group room
Please leave feedback/comments at #quadrix:matrix.org or in the issues at https://github.com/alariej/quadrix (stars welcome :-)
Note: The PR (https://github.com/matrix-org/matrix.org/pull/1348) to publish Quadrix on the matrix.org client list has been submitted more than a month ago, but still awaiting approval. Anyone here can help?
You can now edit aliases in Nheko! This means you can publish your own aliases in the room directory as well as in the room. If an alias can't be used by people (because it isn't listed in the room directory), that alias is highlighted in red with an easy option to fix it (if you have permissions to do so). You can also easily switch the primary (canonical) alias of the room.
With this the feature set I wanted to have for the next release is complete. I wanted to focus on improving moderation and room management capabilities in Nheko and it does now have a very basic sets of capabilities to do so. I do already have plans to expand on them though!
So instead of working on new features, I will be focusing on bug fixes. For a start if someone has the username room, Nheko will now omit the reply fallback to prevent pinging the whole room. The verification window now also should always be big enough to show its contents and we now properly explain to users, why emojis might look different on different devices fixing an issue reported via Twitter... There is also a new icon for the room directory (it is now a building, very punny, huh?).
You have probably seen it, this week Element released Chatterbox, an embedded chat client you can use to build chat assistants, chatbots… or probably other use cases we didn't even think of. It's OSS and really just a lightweight Matrix client. Rick talks about it in greater length with me in today's Matrix Live.
Fix for missing end tokens in sync responses from Synapse >= v1.61.0 (Thanks to Tom Price for !20).
A prettier animation while loading comments.
/ipns/latest.cactus.chat is updated to point to the latest release, so sites linking there should already be using the new version.
Also, while we're here: we're surprised and delighted to so many people using Cactus Comments!
We just crossed 300k guest users registered on cactus.chat (roughly equivalent to 300k unique anonymous users). 🎉
An unfortunate side-effect is that we're having to up our hosting game to keep up with you all - and it's getting a bit expensive on our student budgets.
We set up a donations page on Open Collective, in case any of you would want to help out. ✨🫂
After a long hunt, we've eventually found a significant problem with napi-rs, the layer we use in between Rust and Node.js for the crypto-nodejs, in the way it manages the memory coming from async functions in rust. With that out of the way and the last remaining features implemented, we are on the final stretch into in putting out the first prerelease of crypto-nodejs-bindings—brace for it to come near your next npmjs.com early next week 🤞🤞.
<matrix-room-element/> is a web component (vanilla JS/HTML/CSS & distributed un-minified) that can be imported and inserted in any web page, to display the content of a (public, soon with authentication, and private room support) matrix room.
It is still heavy in development, and still looking for the right patterns (web components, and matrix).
We're trying to make composable components that can be inserted in any web page, and maybe help users use Matrix as a CMS, embedable anywhere (web). Our first use case is on https://libli.org (that now comes with a /login endpoint - alpha) (this event in libli, loading a matrix-room-element with correct room-id/event-id, https://libli.org/thisweekinmatrix:matrix.org/$KsbQ0JsqAXN9g-57M-aXyVohZYM3SZkKKkuUb9dW928).
To explain what it's for: Using the UnifiedPush standard, ntfy enables self-hosted (Google-free) push notifications from Matrix (and other) servers. Especially useful for users of Google-free Android (such as from f-droid).
If you try it, please report any feedback or problems or improvements in #matrix-docker-ansible-deploy:devture.com . Any updates to the scripts or docs may appear first on my branch before being merged into the playbook.
The option for reports to be polled via the synapse admin API (rather than configuring proxy pass-through)
The option to disable the displaying new reports in moderation room (so that you can use the TrustedReporters protection without the abuse reports features)
A !mjolnir rules matching <entity> command to search watched lists.
Glob support to the kick command.
A background queue for kicking (to reduce the load of large glob kicks)
A slight improvement to the performance of the redact command
An improvement to documentation (including dedicated setup documentation)
A new mute action for the since command !mjolnir since 1day mute 100
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Hey all, Synapse
1.62 is out! Let's
have a look inside.
Spam checker improvements
In the past few weeks, the Synapse team has been working with the Matrix.org
Trust & Safety team to help module developers build more efficient protections
against spam. As a consequence of this work, Synapse 1.62 introduces new ways
for modules to communicate the result of actions taken against a specific
message or operation through the spam checker module
callbacks.
Previously, most spam checker callbacks would be expected to return a boolean
indicating whether a specific operation should be qualified as spam. Starting
from Synapse 1.62, modules are now expected to return either
synapse.module_api.NOT_SPAM (to indicate an action is not spammy), or an error
code that is part of synapse.module_api.errors.Codes.
Note that the previous behaviour is still supported but is now deprecated, and
will be removed in a future version of Synapse.
See the upgrade
notes
for a list of the affected callbacks and an example of this change. Note that on
top of the list described there, the check_event_for_spam callback was also
updated with a similar
change
in Synapse 1.61.
Everything else
This release of Synapse includes important performance improvements around
syncing, specifically around handling device lists and notifications.
Synapse 1.62 also introduce a changes of its optional dependency on the LDAP3
authentication provider
module to
v0.2.1
in order to fix an issue with usernames that include uppercase characters.
See the full
changelog for a
complete list of changes in this release. Also please have a look at the
upgrade
notes
for this version.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper, Sami
Olmari, Daniel
Aloni,
Thumbscrew and Hannes
Lerchl.
I had the chance to have Ryan with me for a special edition of Open Tech Will Save Us on Thunderbird to celebrate their 102 release, which includes Matrix support for the first time in a stable release.
We covered many interesting topics, such as the importance of specifying the expected behaviour of clients and servers in a protocol to deliver the best experience to end users (wink, wink, reminds you of something?), why Thunderbird was more dormant and is now vibrant as ever, what they plan for the future. I had good fun and I hope attendees did too! Next episode is going to be at the end of July, but you can already join the
Open Tech Will Save Us room
Audrey Tang, Digital Minister for Taiwan is offering sponsorship for full localisation of Element and other leading Matrix clients into zh_Hant_TW - see https://twitter.com/audreyt/status/1542296087310258176. If you're interested in helping out, please get in touch with Thib and we'll coordinate.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
This MSC is a relatively simple it. Currently widgets are only allowed to specify their capabilities (what the widget is allowed to do, like reading the logged-in user's display name) when the widget is loaded. This MSC attempts to allow that model to be extended to let widgets ask for additional (or fewer) permissions over time.
I can't help but be reminded of the shift in permissions on iOS and Android here in how they shifted from "ask all permissions up front" to "ask for permission for each thing when it's used in the app" :)
New Matrix Intern Usman has just started on his project to prototype Favourite messages in Element Web. He's written a blog post introducing himself and the project: https://yaya-usman.hashnode.dev/outreachy-blog-introducing-myself
This week the Synapse team released Synapse 1.61.1! This is a security release which addresses a high severity vulnerability in URL preview feature. Server administrators are encouraged to update as soon as possible! We have published a blog post explaining the vulnerability and detailing a few workarounds that can be implemented on homeservers which can't be updated right away: https://matrix.org/blog/2022/06/28/security-release-synapse-1-61-1
Other than that we have published the first RC for Synapse 1.62.0 (which was followed today by a second bugfix RC). Synapse 1.62 will feature an improved spam checker API for modules, performance improvements around device lists, more customisation for .well-known client files and much more. Watch this space next week for the full rundown 🙂
This week we released Dendrite 0.8.9 which contains a number of improvements around backfilling room history, amongst other things.
Features
Incoming device list updates over federation are now queued in JetStream for processing so that they will no longer block incoming federation transactions and should never end up dropped, which will hopefully help E2EE reliability
The /context endpoint now returns "start" and "end" parameters to allow pagination from a context call
The /messages endpoint will no longer return "end" when there are no more messages remaining
Deactivated user accounts will now leave all rooms automatically
New admin endpoint /_dendrite/admin/evacuateUser/{userID} has been added for forcing a local user to leave all joined rooms
Dendrite will now automatically attempt to raise the file descriptor limit at startup if it is too low
Fixes
A rare crash when retrieving remote device lists has been fixed
Fixes a bug where events were not redacted properly over federation
The /invite endpoints will now return an error instead of silently proceeding if the user ID is obviously malformed
As always, please feel free to join us in #dendrite:matrix.org for more Dendrite discussion.
Thunderbird 102 with Matrix support is now available for download at https://thunderbird.net/. The Matrix implementation reflects what's been previously discussed in TWIM with some additional bug fixes. You can read more about what's new in our blog post.
Ryan discussed the Thunderbird 102 release as well as the project in general in this week's Open Tech Will Save Us, give it a listen for the latest inside scoop.
Have you ever noticed that some people are just plain @*%&!§%"+? Well, to quickly deal with what they wrote, Nheko now has a /redact @userid:server.name command, so you can redact everything they wrote (as long as it is in the currently cached section of the timeline). Note that you will run into rate limits when using that and Nheko is not yet applying an appropriate backoff in that case.
Similarly, q234rty fixed a lot of cases where icons in Nheko were either blurry or the wrong size. We fixed a few crashes, the room list should now not sometimes store the wrong order of rooms, brausepulver made large avatars cropped locally (since servers don't guarantee any size over 96x96 when cropped and synapse doesn't properly save the full image size in that case) and added a menu entry to copy a roomlink. You can also now define new powerlevels for users instead of using the existing levels in the room and Jason fixed some compiler warnings when a private member struct doesn't have an explicit constructor on some compilers. Nheko also now downloads the full online key backup when you explicitly toggle the switch.
A more noticeable topic might be, that Nheko now requires servers which support the v1.1 API or later and will not allow you to login or register otherwise. At the same time we also enabled support for the shiny and new knock_restricted rule and all remaining groups code was removed (you know, the feature from before spaces were cool).
I also updated Nheko on my work laptop now and since I was quite surprised by how fast it starts now (while my room count is over 900 !), I attached a video of that below for your pleasure.
This week is an exciting time in Element land! We have one feature coming out of beta and another going in: the new search experience is going live with the next release, while video rooms will become available for testing. You can already preview them in the release candidate!
Threads
After investigating ranged read receipts, we are looking at per thread read receipts again as a more practical approach.
Community testing
RC testing done:
Web - MD/HTML support in Space/Room topics
iOS - regression testing of messages, reactions and rooms
Because sometimes you just have to do something else and it might be interesting to some people:
I fixed calls not working when trying to use Element Web with a Conduit server: https://github.com/matrix-org/matrix-react-sdk/pull/8931
I fixed Element Android still using the unstable endpoint (i.e. the one from the MSC, that proposed it) to list aliases: https://github.com/vector-im/element-android/pull/6288
Sometimes fixing something that annoys you isn't even that hard, and since it annoys you, you have some motivation to fix it. I call this Anger Driven Development (ADD). If you have some things that annoy you, I encourage to try fixing them for a bit. It might help out some other people too. If that makes you want to contribute to Nheko, don't hesitate to join #nheko:nheko.im and ask me to help you out with whatever fix you are trying to contribute.
This time I just decided to clean out some issues, that came up in the Conduit development channel and that weren't fixed yet. Turns out it was just a few lines to change and it hopefully makes the Matrix experience better for everyone in the long run. The changes were also quickly reviewed and are merged now. Do you have any such little fixes that no one hears about, but you are proud of them? Please post about them, I'd love to see them!
Are you enjoying Location Sharing but have tired thumbs from tapping every time you need to update your location? Then fret no more! Element 1.8.20 features our new Live Location Sharing feature, now available in Labs.
Live Location Sharing: Now in Labs, we’ve added the ability to send your location in real time to the people and rooms of your choice!
Mark as Read: Quickly mark a single room as read by long-pressing it on the home screen.
Annoying bugs fixed: It’s much easier to tap the names and avatars of room members, voice messages now work better with VoiceOver enabled and it is significantly faster to create a new room.
Over the last few weeks, a significant push on rust-crypto-support for Nodejs has taken place, e.g. this PR. Major parts are available, the infrastructure to create releases ready, however, a pretty rare huge memory allocation that causes a crash in Nodejs on the CI has blocked progress of the team recently. Debugging this costs a lot of time and it is yet unclear, how much of a problem this really is (as the cause stays mysterious), however it has also been seen in the wild on Windows now-- a concerning development.
This week, the team released version 0.10.2. In this version, we added experimental support for native group call signaling MSC3401. We have been playing with it in our internal clients, and we have to say that we are pretty happy with the result! There are still some bugs, as sometimes a new call is creating instead of joining the existing one. But it's looking great!
We also took the opportunity to fix some bugs in call handling. Now the ringtone is properly properly when rejecting a call. Also, when opening the client, we don't trigger a call event anymore if the call end event is in the sync response.
We also did some refactoring for the sync handler, reducing the usage of JSON objects and adding some try catch to handle issues when handling event room updates. We switched to custom CachedStreamController so even if the stream has already been listened to, you can still access the last sent value.
Finally, when sending messages when under unreliable connection, messages could be sent out of order. This is now fixed thanks to the implementation of a message sending queue.
This week's release completed support for resolving, creating, and deleting room aliases. Get the code and contribute at https://git.thisisjoes.site/joe/axon.sh (supports GitHub and GitLab login) and join #axon:matrix.thisisjoes.site for discussion and updates!
Two is better than one! Ascurius joined the synadm team and helps maintain the github repo, code new features and support users on github and #synadm:peek-a-boo.at.
Thanks to all synadm users, feature requesters and contributors, issue submitters and #synadm:peek-a-boo.at members. It's fun to maintain a project and see its community grow each day!
Some features from the latest releases we'd like to highlight:
The local part of an MXID can now be used as the <user_id> argument in various synadm commands. Of course, there is still the possibility to use the full MXID if desired.
We have added a shadow-ban command so that admins now can more easily deal with abusive users: synadm room shadow-ban
The room state API is now supported, try synadm room state
And with the help of that API we created a command to easily generate a list of rooms and corresponding admins and mods: synadm room power-levels
The synadm room search command was adapted to make better use of current Synapse versions possibilities.
Documentation chapter around using synadm together with Synapse instances deployed with matrix-docker-ansible-deploy.
The magic around "retrieval of the own homeserver name" got a massive overhaul. This affects several user and media subcommands.
Have a look at the releases list for more details: https://github.com/JOJ0/synadm/releases
Dept of Events and Talks 🗣️
Matrix Summit Conference Berlin calls for your participation.
With the first Matrix Summit Conference we try to showcase the matrix-eco-system in its whole width and depth.
We are looking for talks and workshops around matrix-related projects and products. We are interested in all aspects of those: From the past to the future, from the moment of the idea, the story of the creation or the vision of the future.
We’d like to understand the principles as well as the technology.
The conference is from people for people, so if you’d like to talk about yourself, your community, your organization, please do.
Showcase yourself and your relation to the Matrix world.
We try to compile a versatile program. We are open to contributions of any length, from 5 minutes(lightning talks) to presentations and talks to workshops and hacksessions up to 5 hours.
We’ll come together to discover, celebrate and enjoy the world of matrix. Also, if you have any arty, cultural or playful contribution in mind, please offer it.
You can enter proposals until 2022-07-22 22:22 (Europe/Berlin), 3 weeks from now in our pretalx
Matrix-summit-berlin-2022 matrix summit space
The #matrix-summit-berlin-2022 space will contain all rooms and subspaces related to the event.
For the organization of the summit we have orga-room. Just come by if you like to know something or help with the summit.
There is a gitlab organisation which contains our codebase and issuetrackers.
Sponsors?
We try to make everything low cost. But we need some money for food, drinks, merch, travel, accommodation. Please contact Yan or write mail, when you can sponsor the event.
Note this is a community event that is not organised by the Matrix.org Foundation.
Meet other matrix users, chat about Matrix, the rest, and everything else, discuss your Matrix ideas, sign each other in persona, and maybe spice the evening with a good mate or beer.
Also when the bbq is lit you may wish you brougth your favorite item :)
Every first Wednesday of the month in the c-base at 8pm ('til the next pandemic).
An oversimplified, easy to implement, embedded live chat widget that allows your website's visitors to send messages seamlessly to your Matrix account.
The widget is Built using Svelte, so everything goes in one nice bundle, its fast and awesome to embed. No need for external libraries or big framework.js files. Just import into your website's structure one JS and one CSS files.
The server uses Golang, the Binary is only 3.2MB in size. Make sure you configure the .env file.
A demo and more information about livematrix can be found here:
https://github.com/osousa/livematrix
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Today we're exceptionally releasing Synapse
1.61.1, which comes
as a security release. Server administrators are encouraged to update as soon as
possible.
This release fixes a vulnerability with Synapse's URL preview feature. URL
previews of some web pages can lead to unbounded recursion, causing the request
to either fail, or in some cases crash the running Synapse process.
Homeservers with the url_preview_enabled configuration option set to false
(the default value) are unaffected. Instances with the enable_media_repo
configuration option set to false are also unaffected, as this also disables
the URL preview functionality.
Server administrators who are unable to update Synapse should disable URL
previews by setting url_preview_enabled: false in their configuration file.
They can also delegate URL preview to a separate, dedicated worker to ensure the
process crashing does not impact other functionality of Synapse.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
The big news from the spec this last week is the release of Matrix v1.3 🎉 (read the blog post if you haven't already)! Roughly three months since the release of Matrix v1.2, this release brings improvements such as knocking on rooms, room version 10, reduced metadata in encrypted messages and the first pieces of aggregations finally landing in the spec proper. And more! See the blog post for the full changelog.
Most of the Spec Core Team has been away this week, thus there has not been much moving forwards. But we do have two new MSCs from @duxovni and @Johennes, which you can view above.
This MSC allows for differentiating between different incoming streams of media coming from a single user by adding a sdp_stream_metadata dictionary to Voice over IP (VoIP)-related events. This is a relatively simply addition with useful functionality, such as allowing a single user to share both their camera feed and screen share at the same time!
1.4.24 released to beta testers which includes support for UnifiedPush and fixes for voice recordings and duplicated messages in the timeline
We're making it easier to opt in to Live Location Sharing by displaying the labs setting within the location sharing flow, no need to hunt down the setting anymore!
We have also fixed some outstanding crashes around opening large images in the timeline and signing out
added previews of room name and topic in tooltips for room icons
added handling for an eventId component of annotation URLs
Along with a number of other minor bugfixes and UX improvements. And, we've added one neat new user-facing feature: inline previews of video and audio annotations. This one is a little hard to explain, but a video is worth ten thousand words:
A set of Rust library crates for working with the Matrix protocol. Ruma’s approach to Matrix emphasizes correctness, security, stability and performance.
Released Ruma 0.6.4 with a bug fix for rich reply fallback generation
Added support for pretty much everything from Matrix 1.3, which was to a large extent just the removal of feature flags for previously-unstable functionality
Thanks to @zecakeh for both implementing most of these features and now stabilizing them in Ruma!
Hey everyone! Guess what? Synapse
1.61 is out! Let's
have a look at it.
Farewell, groups
If you are new to Matrix, you might have not heard of the feature referred to as
"groups" or "communities" (depending on the context). This feature allowed
grouping rooms and users to better represent a community, one of which being
+matrix:matrix.org which used to represent the Matrix community. This may
sound similar to Matrix
Spaces, and it would
make sense since Spaces are meant to be a more powerful replacement for groups.
In Synapse 1.56,
support for groups was deprecated, with a plan to fully remove it in a
later release of Synapse. This has now been done as of Synapse 1.61, and most of the
code supporting this feature has now been removed.
Note that this means that administrators of homeservers using workers can remove
endpoints related to groups from their reverse proxy configuration. See the
upgrade
notes
for more information.
Media retention
A common issue we see homeserver administrators struggle with is managing the
disk space used by Synapse. A non-negligible part of that disk space usage is
dedicated to storing files uploaded by Matrix users, both local and remote.
Up until now Synapse would only provide administrators with limited, manual ways
to manage the media store of their homeserver, via the admin
API.
As of this release, Synapse now allows administrators to define retention
lifetimes for local and remote media. This allows media that hasn't been
accessed in a long time to be automatically deleted, therefore freeing up disk
space. Server administrators wishing to control media retention more finely can
also define different policies for remote and local media.
This feature can be enabled by configuring the media_retention setting, see
the configuration
guide
for more information.
Everything else
This release of Synapse introduces a change in the return value of the
check_event_for_spam spam checker module callback, in order to allow modules
more flexibility in communicating to users why their messages are rejected. This
is part of ongoing improvement works around spam checker callbacks, watch this
space next time for more information!
See the full
changelog for a
complete list of changes in this release. Also please have a look at the
upgrade
notes
for this version.
Synapse is a Free and Open Source Software project, and we'd like to extend our
thanks to everyone who contributed to this release, including (in no particular
order) Beeper, Dirk
Klimpel and Jacek
Kuśnierz.
Hi there! I'm Marco Melorio and I'm participating in this year's Google Summer of Code, under the GNOME Foundation. I'm working on Fractal, the GNOME matrix client, with the help of my mentor Julian Sparber. More specifically I'm working on implementing a media history viewer to the app.
To follow my progress on the project you can check out my blog here. I've already published a small introduction post about me and a first update post which includes a mockup and milestones about the project.
This week the team released Synapse 1.61, which main new feature is media retention. That's right, you can now control how long Synapse keeps media files around, which should help server admins manage Synapse's disk space usage more efficiently. On a different note, this release of Synapse removes support for groups/communities (which was deprecated back in Synapse 1.56), as it has now been replaced by Spaces. Farewell groups, you have served your users well.
See the full Synapse 1.61 release announcement on the matrix.org blog here: https://matrix.org/blog/2022/06/17/synapse-1-61-released
Aside from this, the team is as always hard at work on making Synapse better and more efficient.
New version of Quadrix (1.0.6) available on desktops and mobiles. Mainly bug fixes, plus the addition of a button to deactivate the account on the server (this apparently will be soon mandatory for iOS and MacOS messaging apps). The new desktop version should offer better support for Wayland (tested on Fedora 36, Ubuntu 22.04 and Mobian/Phosh). Repo at https://github.com/alariej/quadrix, project room at #quadrix:matrix.org :-)
Since I skipped the last update, this one is a bit longer :3
First of all there have been lots and lots of updates to the translations! Finnish is now at 100% thanks to the tireless work of Aminda and Lurkki.
Nheko now also shows a nice badge on Unity compatible desktops (like KDE) for unread messages (although that doesn't work properly for multiple profiles being open at the same time due to limitations in that desktop protocol. Thank you d42 for contributing this, may you role a natural 42 every dice throw!
Syldra fixed up the paste behaviour (which didn't properly tie into undo in some edge cases), cleaned up how we find some of our dependencies and made cusor movement more consistent across systems.
Finally we fixed our glare resolution when verifying other sessions, which will be especially noticeable when verifying Cinny, since that responds to verification requests in a different way than I tested before! This should get rid of a whole range of verification issues people might have experienced. As part of our stabilization for the next release, we also fixed a crash on logout because I fatfingered and deleted a return statement, we now send an Element Android compatible height attribute on emotes, properly compile when the C++ version is set to 20, we once again support the current version of the hidden read receipts MSC (so that others can't tell what you read), edits now properly update in replies again, you can close the image overlay again by clicking outside, cancelled uploads properly get removed again, logging out and back in now doesn't mess up your configuration anymore and pinned messages now properly refresh once the events are loaded.
Phew, that was a mouthful. And we are not even done yet!
I spent some time on making Nheko compilation a bit faster again as well as improve startup speed. This might order your room list a bit weirdly for a few days after updating, but should normalize when receiving a few messages in some rooms. With this we now don't need to load the last message in all rooms on startup. This makes Nheko startup now only take 7 seconds on my old laptop (when not doing something CPU heavy at the same time). The remaining startup time is 40% building up the font database and 40% loading the powerlevels for each room. So we do have the chance to speedup startup by probably another 60%. It is unlikely that is necessary though.
When I discovered Matrix, Element was still called Riot. At the time some of the big design changes started happening to make it the Element you know today. One of the sideeffects was that the roomlist was consistently taking up 20% of my CPU on my Laptop, which I used at the time (and am forced to use now again, since I broke my newer laptop). It also used a lot more RAM than it does today. So for that reason I started shopping around for what other clients there are and I found Nheko. Clearly because it isn't a webclient, it had to be faster and use less RAM. Well, it did maybe a bit, but frankly the difference wasn't that much. Especially since at the time it loaded and rendered ALL your messages on startup (kinda). It never removed messages from memory, so it would just continuously render more and more parts of your timeline, which clearly increased RAM usage. It did however never resort the full roomlist, which made it not use a lot of CPU.
Since I didn't know any web development at the time, but I knew some C++, I started contributing to Nheko. At some point I made the roomlist constantly resort, which used quite some amount of CPU, but I quickly fixed that. At the time the most noticeable difference was that my fans didn't spin when using Nheko, but Element did (because of the roomlist sorting, iirc). RAM usage was pretty bad though. So that was one of the next projects, removing all events from RAM and only pulling them from the database as needed. While this means some additional load when switching rooms, it did decrease RAM requirements by quite a bit. Some new features made us still require loading data for every room from the database on startup, which causes quite some noticeable startup delay. The latest changes however got rid of most of that. We now don't need to load the messages from the database anymore. The only thing we still load is a small info object with roomname, notification and member count as well as the power levels of the room.
Since I recently broke my new and fast laptop, I decided to checkout how things changed on my old and slow laptop. Nowadays I am not in 15 rooms anymore, but I am in 900 rooms, but Matrix, servers as well as clients, has also gotten a lot more performant. So in the end I decided to do a little benchmark of where things stand. DISCLAIMER: This is completely unscientific and unfair, so please take those numbers with a grain of salt. Almost no one runs such a slow laptop as I do, so likely your measurements will completely different. Even more, I was video recording the benchmark, which makes both clients a lot slower. Nonetheless it does somewhat mirror my personal experience.
I came to the conclusion that with 900 rooms, Nheko takes about 10-20 seconds to load and be ready for use on my system, while Element takes about 3 to 4 minutes. So basically Element handles 60 times as many rooms about 2x slower than it did back in the day, while Nheko got a bit faster or about the same speed on the same hardware (but still 60x as many rooms). I've attached a sped up video to this post, so that you can compare it for yourself. But since a lot of people ask, I guess the reason is that I wanted to see how fast you can make a Matrix client. I think I somewhat achieved that in the startup time department, but switching rooms still has a loooooong way to go. Also it is just fun to implement whatever you want in a client, since you are the maintainer and none can tell you how bad of an idea it is. That's probably the reason a lot of people start their own clients? (Although I didn't start Nheko, I just wrote too much code and people didn't want to review it anymore.)
That's it, I hope your eyes didn't glaze over with me babbling on about things. See you next time! :3
There have been many improvements to clickable texts in various parts of the application (e.g. when you see “Show more” in the UI) and many other UI tweaks (thank you luixxiul!). As well as UI improvements to colours and layouts in the settings dialog
Fixing a pagination regression because of a synapse update to conform the matrix spec. The fix will be included in the coming release 1.4.22.
Merged a selection of FTUE improvements, including simplifying login if you specify your server as part of your username. This work is not visible yet in the UI.
UnifiedPush is coming! Will be available in Element Android 1.4.22. In the meantime, you can test the feature using a nightly build . This technology will let F-Droid version of the app be able to receive Push and Element Android can stop polling your homeserver in the background, which was consuming lots of battery.
When you send a rageshake, screenshots will not be included by default any more
We also have set up Flipper in the app, to help developers to debug the application. Inspecting the Realm databases, or the network traffic will help a lot when developing new features or fixing issues.
As you can guess by the title, I've started Matrix Wrench one year ago.
Here are the most notable changes of today's birthday release v0.8.0 (2022-06-13):
Added: All pages have URLs to quickly navigate to.
Added: Bulk actions to invite and kick users
Added: The About page now shines some light on the application's features and security.
On other major news, the first UniFFI macro PR, submitted by our very own Jonas, was merged by Mozilla earlier this week 🎉. This is an important milestone as this PR is the opening gate for getting more and more macro-support into UniFFI, eventually replacing the entire UDL-in-between-action. Congrats for this major step forward
The there is a new release that avoids crashing of the bug when the synapse admin is not correctly exposed. For technical reasons (on layer 8) the bug fix is made in two releases. The latest version is now v1.1.7
Four days of Barcamp sessions, presentations, meeting others and Berlin culture!
The Matrix Summit 2022 brings people together who work with Matrix.
Thu, 25th to Sun, 28th August at the iconic hacker space c-base in Berlin.
This event is organised by the local Matrix Meetup community.
While people of every skill level are welcome on all days, we're going to highlight real world use cases of Matrix on Sunday. We're inviting newcomers and other communities to learn about decentralised communication software. In return we're learning how Matrix is already used by the German education and health sectors.
Our Call for Presentations starts next week. The event is going to be bilingual: English and German.
See the schedule:
https://summit2022.matrixmeetup.de/
UnifiedPush is an open protocol for push notifications on Android, independent from Google Services. It is currently supported by FluffyChat, SchildiChat, and now nightly builds of Element.
If you are new to UnifiedPush:
Installing the ntfy app (F-Droid, Play Store) is the easiest way to start receiving notifications for your matrix messages without Google Services.
And if you want to go one step further and self-host a ntfy server, the latest release makes it simpler than ever to set it up on your own server!
For persons already self-hosting ntfy:
The brand new version v1.26 of ntfy server includes a built-in matrix gateway: your homeserver can talk directly to ntfy. If you were self-hosting ntfy with a personal matrix gateway (common-proxies or with a Nginx setup), you can remove the gateway after upgrading to v1.26.
An early-stage home server prototype that communicates in a fully decentralized manner: every user runs their own server locally!
This is achieved with the libp2p library which provides decentralized node discovery, NAT traversal and gossip-based communication capabilities.
The server implements the Matrix API specifications (with many advanced features still missing, like client-side end-to-end encryption), but did little modification on federation side so that it doesn't rely on the centralized domain name system to setup and authenticate other nodes. On the other hand, this means it cannot federate with existing home servers yet, because of the different trust model (domain name/HTTPS vs self-owned cryptographic keys).
Try out the latest Webview client on windows, or on other OSs with Docker&Browser!
anarc.at published a long and thoughtful critique of Matrix over at https://anarc.at/blog/2022-06-17-matrix-notes/ - interesting reading, even if we don't agree with all the points.
It’s another quarter and therefore another spec release! Matrix 1.2 was released back in February, and while a bit late in the quarter this time around we’ve still got some exciting additions coming to you in this release.
Like last time, the speed of these releases might feel a bit quick for developers: fret not if you’re still working on v1.1 or v1.2 implementations. We’re not expecting that implementations update as soon as a new spec release is published, but rather that the ecosystem gradually update over the course of the next few months. Implementations should be aiming for as close to realtime as they can get though, particularly considering v1.4 is expected to have some large features (more on that later on).
Matrix 1.3 sees 14 MSCs get merged, but we can’t possibly go into detail on them all here. We’ve picked some notable highlights and recommend the full changelog at the bottom for a complete idea of what’s been going on.
Aggregations and the relationships made along the way
It’s no secret that MSC1849-style server-side aggregation of related messages have been in the review backlog for a while. We ended up splitting MSC1849 down into more reviewable chunks like MSC2675, allowing us to finally land the first pieces into Matrix 1.3 today.
In the spec there aren’t currently any defined relationships which make use of aggregations or even the rel_type described by MSC2674, but we do expect that v1.4 of the spec will have support for at least threads (MSC3440) and edits (MSC2676), filling the gap. For now, we’ve decided that holding aggregations back until v1.4 doesn’t make a lot of sense, so we have launched them into the world as-is for custom relations or for clients & servers to prepare for what’s expected to be coming in v1.4 of the spec.
To further prepare for threads, we’ve also removed some restrictions of rich replies through MSC3676, thus allowing replies to be constructed with non-text messages like images. Check out the new rich replies module for more information.
Join if you can, or just knock
When we launched restricted rooms in Matrix 1.2 we noted that we forgot to handle a case where someone might want to support both knocking and restricted rooms at the same time. We’ve fixed that with a stop-gap join rule from MSC3787 in room version 10.
The new knock_restricted join rule allows the room to keep its desire to be restricted whilst also allowing members who do not meet the criteria to knock on the room instead. We’ll likely expand on this sort of mixing of join rules in proposals like MSC3386 down the line, however for now this should cover the gap in support. Next up: figuring out how to make encrypted room history available to these new joiners in a safe way.
A thread for next time
Originally planned for this release, MSC3440-style threads are anticipated to be ready for v1.4 (next quarter) with support for notifications and, of course, a whole new way to communicate in a room.
Threads are one of the more complicated features we’ve tried to land in recent history as it has a large impact on a wide variety of the spec: everything from event relationships (fixed in this release) to read receipts to push notifications needs to be worked out to build a system that not only users can understand, but also the developers trying to build it. With this massive surface area we just weren’t comfortable with adding threads to v1.3 as-is, given problems like notifications aren’t yet fully solved.
Alongside threads, we also anticipate that MSC2676-style message editing will be landing in v1.4, finally specifying how to update an event’s contents without having to redact & re-send. Although message editing has been supported for a long time in some clients, we're excited that it will finally become part of the official specification, meaning it can be implemented more widely. Messaging clients are encouraged to give the proposal an early read and possibly even attempt implementation if not already done to help us ensure the system is in a reasonable state for inclusion in the spec, though we do note that feature requests for edits will likely be deferred to a future MSC.
Keep an eye on This Week In Matrix for updates on what v1.4 is expected to include, and how things are progressing.
The full changelog
MSCs are how the spec changes in the way it does - adding, fixing, and maintaining features for the whole ecosystem to use. The blog post can’t cover them all, but that doesn’t make them any less important! Check out the full changelog below, and the Spec Change Proposals page for more information on how these MSCs got merged (hint: they submitted a proposal, which anyone can do - take a look at the Matrix Live episode where Matthew covers the proposal process).
Client-Server API
Deprecations
Deprecate the sender_key and device_id on m.megolm.v1.aes-sha2 events, and the sender_key on m.room_key_request to-device messages, as per MSC3700. (#1101)
Backwards Compatible Changes
Make from optional on GET /_matrix/client/v3/messages to allow requesting events from the start or end of the room history, as per MSC3567. (#1002)
