Rummel came from the third generation of a prominent German musical family, the son of pianist Joseph Rummel (1818–80) and grandson of the composer and conductor Christian Rummel (1787–1849). He studied under Louis Brassin at the Brussels Conservatory, where he won the first prize in 1872, the same year in which he gave first public concert, in Antwerp. During the 1876–77 season he performed in Belgium, the Netherlands, England, France and Scandinavia, and went on to make four tours of the United States (in 1878, 1886, 1890 and 1898), where he married a daughter of telegraph inventor Samuel Morse. One of their sons, Walter Morse Rummel (1887–1953), also became a well-known pianist.
In January 1890 Tchaikovsky declined a request from Rummel to conduct a concert in Brussels that winter, on the grounds of poor health. However, the planned concert did eventually take place in the Belgian capital on 2/14 January 1893, with Rummel himself performing the Piano Concerto No. 1 under Tchaikovsky's baton.
Correspondence with Tchaikovsky
One letter from Tchaikovsky to Franz Rummel has survived, dating from 1890:
One letter from Franz Rummel to Tchaikovsky, dating from 1892, has survived in the archive of the Tchaikovsky House-Museum at Klin — published in a Russian translation only in: (1970), p. 72.