I clicked "follow" on this answer.
It has been deleted since.
Now when I try to unfollow it, I get a message
This post has been deleted; deleted posts can't be voted on
So is it possible to unfollow the answer anyway?
Resolved for now by temporarily undeleting the answer to unfollow it, but the core issue remains the currently still open bug reported on Meta SE, as commented above by Kulfy:
This is a bug already reported on Meta Stack Exchange. The bug exist for questions as well as answers.