TLDR, I’ve spent a fair amount of time exploring this space and it’s really hard and there isn’t just one right way to do it.
The fundamental trade-offs I’ve found are between flexibility and convenience versus run-time performance versus maintainability versus fidelity (precision), with some other subtle nuances thrown in. In my library I focused on flexibility and convenience and low runtime cost, at the expense of maintainability (although I have ways to mitigate some aspects of that) and fidelity.
I have a type for each kind of quantity (i.e. length, time, etc.), and operators to convert to and from real numbers given one of a set of available units. Values are stored internally in SI units, and all of the mathematical operators you’d expect are available (i.e. adding two lengths together works, dividing a length by a time gives a speed, etc.). This design maximizes compile time safety (i.e. you can’t inadvertently assign a speed to an acceleration) and flexibility (i.e. you can add 1 m to 1 ft and get the answer in yards). And since I provide ways of going to and from strings, you can expose this flexibility to your users as well. Since unit conversions and tracking happen only when converting to or from a number, the run time cost of doing math is minimized.
The maintenance cost is that there are a large combination of operations between quantities that need to be supported, and a huge number of units that should be available. The other cost is possible loss of precision, where if you’re doing calculations at either end of the extreme (i.e. in light years or femtoseconds), then storing the values in meters or seconds might incur some loss of precision.
I’ve seen designs that take different approaches to balancing the costs and benefits. For example, you can reduce the loss in precision by making values in every different unit a different type, but that comes at the cost of flexibility (i.e. I can’t add feet and meters any more), or maintenance (i.e. now I have to manually support all of the possible operations between all different quantities and units).
One really cool example is the Haskell library Numeric.Units.Dimensional, which uses some really advanced features of the type system to minimize maintenance costs and run-time overhead, but only supports SI units. The Rust library Dimensioned is pretty cool too.