ZigBee is the future OP. Well maybe not THE future but your future.
I'm talking from having been where you are, at the start of the journey not wanting to spend too much, buying gradually when I had a few quid handy.
I have a Zig Ah Zig Ah stick and it's been flawless. It's basically the same as the Sonoff ZigBee stick everyone says is great, but supplied by another company.
I use it through ZigBee2MQTT which I believe is Open Source.
So the step up is getting a ZigBee stick. Then you plug it in to your HA device (you may have to flash it with a firmware first, and also they work better on a 2m long usb wire to keep them away from your wireless bits of the computer) and install ZigBee2MQTT and configure that.
You already have MQTT so that's already ready.
After that you can pretty much buy ANY ZigBee devices and they'll work. Seriously. None of this Tuya hub for one thing and Sonoff hub for that, all in one.
I personally have IKEA and Aqara buttons working at the moment, one is my doorbell. They're battery powered and work pretty much immediately as compared to the cloud based WiFi lightbulbs I started out with, which always had a small, but noticeable lag
First ZigBee device I bought, to test that the stick worked, was an Aqara Cube. Have a Google, they're really cool and totally pointless, everyone should have one.
But then you have motion sensors (buttonless lights), door sensors (also buttonless lights, good paired with motion sensors to keep the lights on when you go for a shit), leak sensors (I have a post on here about my bath-is-run automation using leak sensors), temperature sensors (handy for automated heating/AC), and I'm sure there's more I've forgotten.