Rudolph was born in Merritt Island, Florida. After his father, Robert, died in 1981, he moved with his mother and siblings to Nantahala, Macon County, in western North Carolina. He attended ninth grade at the Nantahala School but dropped out after that year and worked as a carpenter with his older brother Daniel. His mother believed in the racist Christian Identity movement and instilled its ideology in him.
After Rudolph received his GED, he attended Western Carolina University in Cullowhee for two semesters in 1985 and 1986. In August 1987, Rudolph enlisted in the U.S. Army, undergoing basic training at Fort Benning in Georgia. He was discharged in January 1989 while serving with the 101st Airborne Division at Fort Campbell in Kentucky, due to marijuana use. In 1988, the year before his discharge, Rudolph had attended the Air Assault School at Fort Campbell. He attained the rank of Specialist/E-4.