The discovery of Little Red Dots (LRDs)—a population of compact, high-redshift, dust-reddened galaxies—is one of the most surprising results from JWST. However, the nature of LRDs is still debated: does the near-infrared emission originate from accreting supermassive black holes (SMBHs), or intense star formation? In this work, we utilize ultra-deep Chandra observations and study LRDs residing behind the lensing galaxy cluster, A2744. We probe the X-ray emission from individual galaxies but find that they remain undetected and provide SMBH mass upper limits of ≲(1.5–16) × 106M⊙ assuming Eddington limited accretion. To increase the signal-to-noise ratios, we conduct a stacking analysis of the full sample with a total lensed exposure time of ≈87 Ms. We also bin the galaxies based on their stellar mass, lensing magnification, and detected broad-line Hα emission. For the LRDs exhibiting broad-line Hα emission, there is a hint of a stacked signal (∼2.6σ), corresponding to an SMBH mass of ∼3.2 × 106M⊙. Assuming unobscured, Eddington-limited accretion, this black hole (BH) mass is at least 1.5 orders of magnitude lower than that inferred from virial mass estimates using JWST spectra. Given galaxy-dominated stellar mass estimates, our results imply that LRDs do not host overmassive SMBHs and/or accrete at a few percent of their Eddington limit. However, alternative stellar mass estimates may still support that LRDs host overmassive BHs. The significant discrepancy between the JWST and Chandra data hints that the scaling relations used to infer the SMBH mass from the Hα line and virial relations may not be applicable for high-redshift LRDs.