What do to about a clogged dishwasher drain?

Our dishwasher has been leaving what looks like bits of paper in the bottom of our plastic cups and sometimes other dishes for a couple months now. We think the drain may be clogged, but we don't know how to unclog it. (Can't use liquid plumber because that would put toxic chemicals on our dishes.) We've been careful to remove the labels from anything we put in.