How do you respond when someone teases you continually and annoys you?

Don't be so thin skinned, and if you are hanging up on someone, tell them, do it, and don't apologize for it. You told him you were going to hang up.

If he's annoying you, annoy him back by humoring his accusations, when he jibs at you and accuses you of taking his position when he leaves, say, "Yup, so when are you turning in your notice of resignation?" He sounds like he's just looking for his ego to be stroked, so he's fishing for compliments by always talking about you taking over when he leaves. It's a somewhat passive aggressive move he's using on you. He'll probably take the very thing that you've said and used it in his meetings with his superiors, meaning, "So and So isn't capable of doing my job"....

So humor him, since he wants to annoy you, annoy him back, because he's obviously insecure about his position and feels threatened by you. The less you say, when he says that, the better, he's trying to get a response out of you, and you probably have a non confrontational personality, perfect for his BS. Don't allow him to make you uncomfortable or take you out of your element with his foolishness. Respond to his foolishness by saying Yes, or I sure do, and then keep it moving.

And absolutely do not apologize for it.

Yep you are, and need to grow some skin. This man is obviously bored and because you react to him it becomes amusing and he will keep on doing it. The best thing you can do is have as little to do with him as possible. There is such thing as bullying in the worse place and another approach is rather than letting it stress you out go and tell someone without caring how you may look as a result, you’re better off in a comfortable environment.