I can't seem to find the source right now (so take this with a grain of salt), but someone did check the Lemmy codebase to look into this scenario - if an instance has downvotes disabled, it won't propagate incoming downvotes.

I know that on my instance which does have downvotes enabled, if I check out any post from Beehaw (who does have downvotes disabled) there are zero downvotes on any comments as far as I can see.

Now I'm assuming this would only apply on communities hosted on that (or any downvote-disabled) instance, using my previous example if someone from Beehaw were to comment on a community originating lemmy.ml, I believe they could still have their comments downvoted since lemmy.ml would be "hosting" the comment, so to speak.