Charles “Sweet Charlie” Brown was born in Canton, Mississippi, on February 24, 1936. Brown's parents, Ruby McClure Brown and Reginald Brown, Sr., moved to Chicago when Brown was still young, and playing basketball in Washington Park became the center of his activity. After attending grade school at Betsey Ross, Brown went on to attend DuSable High School.
In 1954, along with teammates Karl Dennis, Bobby Jackson, Paxton Lumpkin, and Shellie McMillon, Brown was part of the DuSable Panthers basketball team, which became the first all-black team to reach the finals of the Illinois state high school basketball championship. The Panthers finished second that season, with their only loss coming in the final game. Following high school, Brown briefly attended Indiana University before transferring to Seattle University; there, he joined basketball legend Elgin Baylor on the court, and Brown’s game winning shot against UCLA in the Elite Eight propelled Seattle to the Final Four in the 1958 NCAA tournament. The following year, Brown was named as an All-American.
Following his graduation from college, Brown played in the Amateur Athletic Union (AAU) on teams in Seattle and Chicago. Following his retirement from basketball, Brown worked as a community relations executive for the Jewel Grocery Corporation. He has since retired, but his passion for basketball remains. In 1994, Brown officiated over the Class AA Illinois state championship game, and the assignment director of referees for the Chicago Public League. In 1995, Lettuce Entertain You and Rich Melman began funding a senior basketball tournament, now known as the Windy City Shootout; Brown served as the president from the beginning of the tournament, which was aimed at players fifty and older from around North America.