The concepts of “the language organ” and “the language acquisition device” advanced by Chomsky and others in the tradition of Generative Grammar are controversial in part because their neurobiological instantiation is unclear. Here I address this by reviewing recent evidence from brain imaging and lesion-symptom mapping in aphasia. I propose a model for how the language organ develops in the brain in the context of the recent thesis of the architecture of the human language faculty and its evolution by Berwick & Chomsky, 2016. In this proposal, an innate syntactic computational system combines with domain general sequencing systems, which become gradually specialized for speech externalization during language development.