Background: We aimed to assess the lifelong impact of extreme stress on people who survived the Holocaust. We hypothesised that the impact of extreme trauma is detectable even after more than 70 years of an often complicated and stressful post-war life. Methods: Psychological testing was performed on 44 Holocaust survivors (HS; median age 81.5 years; 29 women; 26 HS were under the age of 12 years in 1945) and 31 control participants without a personal or family history of the Holocaust (control group (CG); median 80 years; 17 women). Magnetic resonance imaging (MRI) using the 3T Siemens Prisma scanner was performed on 29 HS (median 79 years; 18 women) and 21 CG participants (median 80 years; 11 women). The MRI-tested subgroup that had been younger than 12 years old in 1945 was composed of 20 HS (median 79 years; 17 women) and 21 CG (median 80 years; 11 women). Results: HS experienced significantly higher frequency of depression symptoms, posttraumatic stress symptoms, and posttraumatic growth, and lower levels of well-being. The MRI shows a lifelong neurobiological effect of extreme stress. The areas with reduced grey matter correspond to the map of the impact of stress on the brain structure: insula, anterior cingulate, ventromedial cortex including the subgenual cingulate/orbitofrontal cortex, temporal pole, prefrontal cortex, and angular gyrus. HS showed good adjustment to post-war life conditions. Psychological growth may contribute to compensation for the psychological and neurobiological consequences of extreme stress. The reduction of GM was significantly expressed also in the subgroup of participants who survived the Holocaust during their childhood. Conclusion: The lifelong psychological and neurobiological changes in people who survived extreme stress were identified more than 70 years after the Holocaust. Extreme stress in childhood and young adulthood has an irreversible lifelong impact on the brain.