Annotations is not detecting when the change is made by renaming, move files or trivial changes as trailing spaces.
This is something git gui blame has solves as it shows the original commit and the commit that changed it to the last state.

So normally you want to know the original commit/committed, not the one who moved it.

It will be ok to just add this funcionality in the same way git gui blames is doing, with a second column and also adding the commit to the popup that shows in the moused pointer