At night, TST Bus Terminal near TST East MTR. Very few ppl. THere is a disabled's toilet and very convenience. After changing, you may walk alone the harbour side. Of course, if you worry, don't wonder into the core TST East.
On the other hand, the Peak is a good choice during daytime, few locals, and tourists don't bother you. but because many ppl take photo there, you may worry.