Is there any way to teach my 4 month old to stop taking the pacifier out of his mouth?

Usually he won't go to sleep unless we give him the pacifier and rock him. (and NO I will not do cry it out) But he has now discovered that he can take the pacifier out of his mouth; and so he does it almost constantly but I don't think he really means to; because he starts to cry when he realizes the paci is gone and it takes a while to get him to sleep. Is there any way to teach him not to do it?