If you don't clear the cache, somebody could forensically examine your hard drive, and infer what websites you've been to.

The tricky thing is, even if you clear the cache on exit, the files still exist on the hard drive or SSD. And still can be recovered forensically. It's better to not write them at all to disk if you're worried about privacy

If you want to browser that doesn't store anything on disc, look at the tor foundation browser, or the mullvad browser. Both code bases do everything they can to prevent things from being written to disk even temporarily.

You probably want to add to your post what kind of device in what kind of environment we are talking about.

Is it a device in your own home where you live alone or something you carry around? Is the device storage encrypted?

What are you worried about? Local access or some sort of data leakage when you revisit a site?

Is this really necessary since if it is your own device

Try librewolf

In my opinion there attack surface about cache of browser which can lead to privacy leak https://portswigger.net/web-security/web-cache-poisoning

Let's look at this from an even more practical use case. The cache can become very large. To the point of it interfere with loading other programs and even itself. You absolutely should clear it even if you don't care a privacy because the benefits are minimal