constant transcoding is the issue. Doesn't really seem to matter how powerful the server is. I've found the best option is to standardise on a single codec format and a set of bitrates based upon resolution.. EG: older DVD based 720P I use AV1 at 800Kbps.... Then all my clients will direct play
Transcoding the library outside of Jellyfin allowed me to use an 4th gen i3 with 12GB RAM for the jellyfin server itself as it now just streams everything, no transcoding on the fly whatsoever
suggest buy an apple tv, and grab FireCore's Infuse player (free with a sub for other codecs) Jellyfin has a plugin, heck the infuse player will offer to install it for you.. I direct stream everything 264, 265, AV1 without issue from an I3