You can log actual time when you are conducting flight SOLELY BY REFERENCE TO THE INSTRUMENTS. Regardless of if you are on an IFR or VFR flight plan or are conducting flight in IMC or VMC conditions.
Here's an interpretation from the FAA Chief Council:
Dear Mr. Carr:
This is in response to your letter asking questions about instrument flight time.
First, you ask for an interpretation of Section 61.51(c)(4) of the Federal Aviation Regulations (FAR) regarding the logging of instrument flight time. You ask whether, for instance, a flight over the ocean on a moonless night without a discernible horizon could be logged as actual instrument flight time.
Second, you ask for an interpretation of Section 61.57(e)(2) of the FAR, noting that Advisory Circular 61-65A, Certification: Pilots and Flight Instructors, seems to contain advice contrary to your understanding of the rule.
As you know, Section 61.51(c)(4) provides rules for the logging of instrument flight time which may be used to meet the requirements of a certificate or rating, or to meet the recent flight experience requirements of Part 61. That section provides in part, that a pilot may log as instrument flight time only that time during which he or she operates the aircraft solely by reference to instruments, under actual (instrument meteorological conditions (imc)) or simulated instrument flight conditions. "Simulated" instrument conditions occur when the pilot's vision outside of the aircraft is intentionally restricted, such as by a hood or goggles. "Actual" instrument flight conditions occur when some outside conditions make it necessary for the pilot to use the aircraft instruments in order to maintain adequate control over the aircraft. Typically, these conditions involve adverse weather conditions.
To answer your first question, actual instrument conditions may occur in the case you described a moonless night over the ocean with no discernible horizon, if use of the instruments is necessary to maintain adequate control over the aircraft. The determination as to whether flight by reference to instruments is necessary is somewhat subjective and based in part on the sound judgment of the pilot. Note that, under Section 61.51(b)(3), the pilot must log the conditions of the flight. The log should include the reasons for determining that the flight was under actual instrument conditions in case the pilot later would be called on to prove that the actual instrument flight time logged was legitimate.
<letter edited for brevity>
Sincerely,
/s/
John H. Cassady
Assistant Chief counsel
Regulations and Enforcement Division
My take is that if you determine that you are unable to continue flight visually due to poor horizion references (uneven cloud layers/haze) or no horizion at all (dark night over featureless terrain), and must fly based on sole reference to instruments, then you can log actual.