I've been there a bunch, never had a problem with safety & security. If you plan to venture out at night, especially after some adult beverages, that's when you are most vulnerable.
Most unfortunate cases happen with a solo individual walking back after a few drinks. Yes, there have been things that have happened right in a daytime coffee shop. Just turn your S.A. up a few notches and you 'should' be fine.