No you don't. You press a button and the speaker clicks off and the Direct Connect works just like a regular fone call but in simplex mode like an old radio telephone. Actuall on my i95 if I am in a DC conversation and I fold open the fone it automatically goes from the speaker to the regular fone type deal. Personally, I think people who talk loudly on the pnone like they are in their living room when they are in a resturaunt or some place is rude, speaker phone or not.