this post was submitted on 01 Sep 2023
254 points (96.4% liked)

Programming

17495 readers
149 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[โ€“] BatmanAoD@programming.dev 2 points 1 year ago (1 children)

If I explain myself or add nuance, it won't be a "hot take" anymore, but here goes...

  • I've mostly worked on small teams, but have maintained fairly large codebases (I think the largest single repo I've worked in was about 1.5 million LoC). I've also maintained software used by other teams, though in fairly small software organizations (under 100 developers total).
  • I've written code with 100% unit test coverage, and code with almost no coverage. The 100% coverage was in a library that was intended to be core functionality shared across multiple teams.

I definitely agree that they can be useful as both usage examples and as a way to enforce some rules (and consistency) in the API. But I'm not sure I'd go so far as to call that a "spec", because it's too easy to make a breaking change that isn't detected by unit tests. I also feel that mocking is often needed to prevent unit tests from becoming integration tests, but it often hides real errors while excessively limiting the amount of code that's actually being exercised.

I also think that actual integration tests are often a better demonstration of your API than unit tests are.

I haven't used any of these methods as much as I'd like to, but I suspect that each of them is strictly more useful than standard unit testing:

  • contract tests (arguably just a form of unit testing, but not the way I've generally seen it done)
  • property-based tests
  • mutation tests
  • preconditions & postconditions (with language support)
[โ€“] Elderos@lemmings.world 0 points 1 year ago

Makes a lot of sense. I figure contract tests is more or less what I have been doing then.

I think there is that misconception that unit tests are about validating each line of code and preventing logic bugs. Though obviously you understand that this isn't just about that, or not at all about that I would argue. Unit tests won't prevent accidental breaking changes, but you can at least add new tests every time this happen, so you're at least guaranteed that the next maintainer won't be doing the same mistake.

In an ideal world we could probably have nothing but integration tests, but in my experience if you only do integration testing you end up working really hard maintaining tests that are prone to break and costly to run. Unit tests are cheap to code and cheap to run, it is a great place to enforce your "contracts" with tests that are theoretically immutable. After those tests are out of the way, you can focus only on the actual interaction between your systems with the more expensive tests.

Anyway, you have a good take I am just blabbering. This is based on my personal experience as someone who only cared integration tests but was later converted by a person much smarter than I am. And then later on by joining a team that exclusively did integration testing, hundred of tests. It was hell (imo), we had to change dozens of tests for every little bit of modification. The rest of the team seemed fine with it though. We rarely shipped bugs but progress was incredibly slow for a module of such low complexity. Testing wasn't the only issue with that codebase though.