The neurophysiological response during comprehension of sign language has been studied using functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) since the advent of neuroimaging. Deaf signers have been shown to recruit similar persivyliyan regions for sign language processing as those identified in studies on verbal language. To date, the neuroimaging literature on sign language has only been reviewed qualitatively and the involvement of right-hemispheric homologues of left persivyliyan language regions remains subject to debate. We recently used Activation Likelihood Estimation to observe spatial convergence across studies on sign language comprehension in deaf signers (Trettenbrein, Papitto, Friederici, & Zaccarella, in preparation) and found that sign language comprehension recruits widely distributed bilateral fronto-occipito-temporal networks, with the largest cluster in left inferior frontal gyrus (IFG) spanning Brodmann areas (BA) 44 and 45 (i.e. Broca’s region). Mass-overlap analysis using cytoarchitectonic masks revealed the involvment of left but not right BA 44 in sign language comprehension. To identify regions associated with processing of human actions and movements irrespective of the presence of linguistic information we compared our dataset to an independent set of neuroimaging studies of action observation (non-linguistic manual and bodily actions) in hearing non-signers. Activity in right IFG during sign language comprehension converged with activation found in hearing non-signers when observing manual actions. This suggests that Broca’s region is involved in processing of linguistic structure and meaning of sign language stimuli whereas right IFG activity may be specific to processing language in the visuo-gestural modality. Meta-analytic connectivity modelling confirms that voxels in the observed activation mass in left BA 44 and 45 co-activate with the core and extended language network in hearing non-signers during studies of language processing. These results point to Broca’s region as a modality-independent hub in the language network that is involved in language processing regardless of the modality of language use (spoken, written, or signed).