I don't think it's currently possible.
Android apps are able to open links via the Open supported links
functionality within the app's App info
screen (where you would go to manage the app's permissions, notifications, etc.). You can even toggle the functionality by various domains or subdomains per app.
However, I think the app needs to declare what links it supports. I'm not aware of a way to add supported links for the user to toggle on/off once the app is installed.
Assuming that's accurate, it also leads to the question - how would Lemmur handle various Lemmy instances? It could obviously include https://lemmy.ml or other large instances, but I doubt the developers want to support every small/private instance.