I would rather just have a singular name like “squajibbles” for milimeters and memorize an intuitive sense of what that is. I realize I can do that with the word “milimeters” too but my brain sometimes gets stuck on unpacking the math.

This is, in fact, exactly what metric users do in their daily lives... We don't do math in our heads every time we measure something. We know from experience how large all the units are and pick the one that's appropriate for a given situation, just like you do.

When you measure something using inches, you don't then say "it's this many 1/36ths of a yard" unless you specifically need to convert it into yards for some reason.

Similarly, when we measure something using millimeters, we don't say "it's this many 1/1000ths of a meter". It's just a millimeter. Don't get hung up on the prefix, just ignore it and treat it as a unit of a particular size.