Rob Yeung's Answer:
You're right to believe that this rudeness can't continue. It will undermine your authority if you allow him to carry on this way. Also, other employees may start to feel their morale gradually wearing away when they see this person getting away with it when they are successfully making efforts to keep their behaviour in check.
You say that you are "making some headway" so I'm sure you already suspect this isn't an issue that can be corrected overnight. I always believe that the best way for dealing with such an issue is to have a straightforward, one-to-one discussion with anyone who may be causing problems.
I run workshops in which I try to teach managers about "both/and" thinking rather than "either/or" thinking. Traditionally, most of us have a tendency to engage in "either/or" thinking, i.e. to see the world in black and white terms. When we have a difference of opinion, we often believe that either "I'm right (and you're wrong)" or "you're right (and therefore I'm wrong)".
However, a more productive way of looking at many conflict situations is to accept the possibility that both you and I may be right at the same time. I can see a situation one way and you may see it in a different – but equally valid – way.
Before raising the issue with him, may I suggest that you try to find the third perspective on this situation. Imagine that you're taking part in a reality show and that both you and your employee are being filmed by secret cameras. Then imagine that an impartial observer – perhaps a psychologist or the viewers at home – are trying to describe the story from as objective a viewpoint as possible. What would neutral observers say about the situation?
By looking for this third perspective, a helicopter view of the situation, we can find a more neutral tone with which to approach a discussion. Rather than seeing it as a problem with his rude tone of voice, we can phrase it as a difference between the two of you, which will come across as less judgemental and more helpful a starting point.
Of course you feel that the facts are on your side. He's being rude and you're his boss. But simply pointing that out isn't necessarily going to make the issue go away. You say that he hasn't had a manager for about three years, so there are probably many issues running around in his head.
Giving him opportunities to get them off his chest – perhaps over more than one discussion – may help him to feel better about you and the work. So say something like: "You and I seem to have different priorities. I get the impression you're not happy with the way I'm managing you. What can we do to sort this situation out?"
Then be prepared to listen. Don't try to defend yourself initially. Let him do all the talking. Allow him to pour out his thoughts, his feelings, his perhaps shattered dreams and frustrations. And still don't say anything. Tell him that you want to think about it. Then go away to mull over your response before you come back. That way you can respond in a calm, considered manner rather than jumping to conclusions.
Remember that it's your responsibility as a manager to lead an effective, productive team – not necessarily to be voted as the most popular person around. You may not become his best friend, but you must at least earn his respect.