IME women nowadays get offended when you call them ma'am, so I chose the lesser of two evils. 
As of now, as I have to told by dealing with agencies on my Mothers' behalf.....Older persons like to hear "ma'am" or "sir" it shows respect for their age. Otherwise use their name or leave it non-sex identifiable. As well, never mention race when describing someone...for what ever reason, it would be or could be taken as racist.