If push comes to shove just remember to never "refuse to fly" you tell them you "can't accept that asignment" There is a huge difference in the hearing. Refusing to fly is insubordination. Unable to accept an assignment is very different.

In your case I'd say your time of notification started at 4 am. You should review the FAQ for FAR117. Although its not law yet airlines are strongly encouraged to implement them earlier and comply with whatever provisions they can ASAP.