This afternoon I departed VFR from a Class D airport underlying Class B airspace. Tower cleared me for take off and I was instructed to fly the standard VFR departure. On departure I climbed up to 3000 feet and tower called me and asked me to descend back down to 2000 feet to avoid Bravo airspace. As soon as I reached 2000 feet the tower instructed me that I was clear of class Bravo airspace and could climb on course. I restarted my climb and as I was climbing through 4500 I was switched over to departure control. The departure controller gave me a sqawk code. Departure called radar contact said cleared to climb through Class Bravo airspace. Then the controlled called me to back to tell me that next time I needed to have a better understanding of their airspace. I responded that I was cleared to climb by the tower and departure responded that only they have control of the airspace. I realized that I had entered class B airspace prior to tower switching me over to departure control. I apologized to the controlled and informed them it would not happen again. I did not receive a "possible pilot deviation" notification and I was not given the number to call. I was a few miles to the East of the main airport and did not hear any evidence of a loss of separation. As a precaution I am filing a ASRS report.
My question is since departure cleared me to climb through class B airspace and did not notify me of a possible pd or give me the number do I need to be concerned about a potential FAA investigation?