As an first officer I used special VFR quite a bit in the state of Alaska at a towered airport that was was not in radar coverage. Often there would be 15-30 miutes delay to wait for your IFR departure or arrival clearance, when special VFR got you out or in a little faster. And the airports we served from that airport were not equipped with instrument approaches anyway.
That being said, even though I was in a instrument eqiupped, two crew aircraft where both pilots were familar with the terrain, I had a few experiences I would care not to repeat in SVFR conditions. The crap can really hit the fan quick... I learned enough from those experiences that when I became captain, I never used SVFR again. I took the extra gas picked up my IFR clearance and waited patiently so that I could have a nice margin of safety in my departure or arrival. IMHO flying low in reduced visibility conditions is asking for trouble, especially if you decide to make it a regular occurence or are not instrument rated.