I'll do my best to explain:

Firstly, not all code executed on an open source OS needs to be open source. For example: Epic Anti-Cheat, which comes with a Linux-compatible mode, is fully closed source. So right off the bat we're going to put to bed the notion that somehow the platform of choice makes it easier for bad actors to pull apart and examine anticheat software.

Secondly, yes, there is a problem with cheaters being able to hide from anticheats on Linux. This is because on Windows it's relatively easy to run kernel-level code via drivers -- this is why most anticheats require admin permission to install a monitoring driver before the game will run. The anticheat is effectively rootkitting your system in order to circumvent other rootkits that may be concealing epic cheatz.

On GNU/Linux, almost all device drivers come prepackaged in the Linux kernel, so there's no direct equivalent to the Windows approach of allowing users to install third-party code into the most protected rings of the OS. It's still possible through the use of kernel modules (see NVIDIA drivers), but as evidenced by how annoying it is to use NVIDIA devices on Linux, this is a huge PITA for both the developer & the user to deal with.

So that's the rub. On Linux, anticheats just have to trust that the kernel isn't lying. This has been a perpetual thorn in the side of developers like Google, who'd really really like it if they could prove beyond a shadow of a doubt that a given Android device is not rooted (see SafetyNet). Google's solution to this has been to introduce hardware-backed attestation -- basically a special hardware chip on the device that can prove that the kernel software has not been tainted in any way.