Purpose The purpose of the study was to evaluate cerebral morphological changes in temporal lobe epilepsy with hippocampal sclerosis (TLE-HS) and their relationship to the cerebellum. Methods The study cohort included 21 patients with intractable TLE-HS (14 left-sided, 7 right-sided) and 38 healthy controls (HC). All patients later underwent anteromedial temporal lobe resection. All subjects were examined using a 1.5-T magnetic resonance imaging (MRI). Volumes of distinct cerebral and cerebellar structures were measured using voxel-based morphometry. The structural covariance of temporal lobe structures, insula, and thalamus with cerebellar substructures was examined using partial least squares regression. Results Morphological changes were more significant in the group with left TLE-HS when comparing left-sided with right-sided structures as well as when comparing patients with controls. The gray matter volume (GMV) of the temporal lobe structures was smaller ipsilaterally to the seizure onset side in most cases. There was a significant amygdala enlargement contralateral to the side of hippocampal sclerosis in both patients with right and left TLE-HS as compared with controls. Selected vermian structures in patients with left but not right TLE-HS had significantly larger GMV than the identical substructures in controls. The structural covariance differed significantly between patients with left and right TLE-HS as compared with HC. The analysis revealed significant negative covariance between anterior vermis and mesial temporal structures in the group with left TLE-HS. No significance was observed for the group with right TLE-HS. Conclusion There is significant asymmetry in the GMV of cerebral and cerebellar structures in patients with TLE-HS. Morphological changes are distinctly more pronounced in patients with left TLE-HS. The observed structural covariance between the cerebellum and supratentorial structures in TLE-HS suggests associations beyond the mesial temporal lobe structures and thalamus.