ignition lock housing... stuck on harness (on 1985 300SD)

I'm in the process of replacing the ignition lock housing.  I've done
most of the work (as you can see from picture) but I cannot remove the
wire harness.  I've tried pulling with a pair of pliers with no
success.

http://www.acequality.net/temp/iglock.jpg

Is there a trick to removing the harness?  Or should brute force do
the trick?  I did see three little notches where plastic is seen
through, I've tried pushing the clips in but they do not appear to be
harness plastic.

Any body know how to separate the two?