Most of the time I wait until we're at cruise to go to speaker and usually go back on headset around FL280 on the way down. Personal preference, but I struggle to hear the speaker and/or FO at 320kts down low. (I'm not old or hard of hearing) I use a clarity aloft so if the other guy wants to go to speaker at 180, it's easy to pull an earbud out.