I know my husband does from time to time, and I know a few others who have used mine over the years including my dorm mate's boyfriend.
I think its sexy that a guy could get off just by my scent or taste while imagining me in them or out of them.
As somebody else said, looking at your avatar, I can see why your husband would like to use yours!!!
I'm jerking-off at the moment smelling a pair of my wife's undies. They smell so sexy!!!