Controllers have the responsibility to ensure accuracy of read backs (was trying to get the 7110.65 reference but this computer isn't playing along). If the pilot acknowledges with "roger" and deviates from the clearance, the pilots at fault, if the pilot reads back the clearance and it is not corrected by the controller, the controller is at fault. That said, if ever in doubt about it controllers will submit an ATSAP and pilots should submit ASAP (or NASA, etc). In the enroute environment, deviations get researched by the regional office and the course of action taken is initiated, normally, outside the facility.