Born and raised in Beaufort County, North Carolina, James H. Smallwood continues to live on what remains of the ancestral piece of property that his great-grandfather purchased in 1871. As a child, he worked that land with his father and grandfather. He went on to graduate from P.S. Jones High School in 1962 and, in 1965, joined the Army and served time in Okinawa during the Vietnam War. When he returned home, he eventually did construction work, worked for Texas Gulf, and then Werhauser, where he retired. Today, Smallwood is the historian for the P.S. Jones Museum of African American Education.