As you may have already figured out, there are many factors to consider.

Directly from the Airbus procedure:
-At takeoff
-If before V1
The takeoff should be rejected only if significant airspeed variations occur below indicated V1 and the pilot decides that there is sufficient runway remaining to stop the airplane
-If after V1
Call- “Windshear TOGA”
-Thrust Levers TOGA
-Reaching VR ROTATE
-SRS Orders FOLLOW

The reality is in most cases the V1 and VR speeds are higher than Vmu (Mininmum Unstick) as they are usually optimized for second segment climb performance. Put simply, the airplane will fly at speeds lower than V1 and Vr.

The only other factor that I consider is the calculated distance remaining after a reject is performed at V1- depending on the runway (say a 12,000' runway) I've seen as much as 3,000' feet remaining. As the Airbus procedure says, if you 'believe' that it is possible to stop, then stop. If you do not believe you will stop (e.g. KLGA) then GO!!