Direct perception and theory–theory approaches to social cognition are opposed with respect to whether social cognition is inferential. The latter argues that it is inferential, the former that it is not. This article argues that the opposition in terms of inference is mistaken. A sense of inference is specified on which social cognition can be inferential and directly perceptual. Arguing for inferential social cognition does not commit to a defense of indirect social cognition if inferential access to other minds can be direct. Contrary convictions are symptomatic of working with too simplistic a notion of inference. The dispute between direct and inferentialist social cognition is one in which both sides can be right. The argument, then, is that inferentialism should not be called on to witness in favor or disfavor of advocates of either direct or indirect social cognition.