We generally take great pains to ensure that the line is clear before making a drop, and in most cases, aren't authorized to drop unless the line is clear. During urban interface situations (houses among the burning fuels), it's not an easy determination to make. Removing people from their homes is a law enforcement issue, and there isn't the same accountability as with fire crews.
I made two attempts to drop on a structure about 17 years ago, and had to go around both times when someone emerged from the garage, bucket in hand. Law enforcement was called; the people wouldn't leave, and we moved to another target and protected it, instead. I suspect that the structure was lost.
The last time I hit people with retardant on a fireline, I got stuck with a 17 page report, and had to give presentations for three years after. Royal pain. Not again, thanks.