This is the general gaming rule typical at a casino:
If a single card is exposed by the dealer in dealing the hole cards,
the dealer must—
(a) complete the initial deal; and
(b) exchange the exposed card with the card that was to be the
first burn card before the flop; and
(c) give all players the opportunity to view the exposed card;
(d) use the exposed card as the first burn card.
If two (or more are exposed) its a miss-deal and a complete re-shuffle / next deck.
So in your situation it all hinges on the definition of 'exposed'.
The general definition taken is that a player is able to 'announce' the card - not a range of possible cards. Saying its a face card is not enough. You could argue that even saying "red ace" is not enough, because that does not expose the specific card, just narrows the range of the possibilities the card can be. There is grey area here. Generally colour and rank is accepted.
I can't think of any time I have ever seen someone accept 'face card' alone. This does not expose the card, it merely reduces the range to one of 16 cards... I don't think anyone on the floor would ever accept anything less than at least the face value. The most common minimum is face value and colour. Suit is generally not required because it cannot be read by the text value.
If he felt that strong about it, he should have requested that the card be declared exposed. At which point the floor would have asked him to name the card. And a decision been made as to if his description has announced the card to the point of exposure.
As soon as an action starts, specially a bet started for the hand, any claim he had is now out the window. You could maybe argue that action be allowed to come around to him. But once he acts in any way, any last hope he had goes out the window.
Complaining about it after the hand is ridiculous.
I'd say you played it fine. It would be absurd if players started to announced they saw a "mostly white - non face card" for example.