In January, 1846, he went to school at Allesley, near Coventry, under the Rev. E. Gibson. He seldom referred to his life there, though sometimes he would say something that showed he had not forgotten all about it. For instance, in 1900 Mr. Sydney C. Cockerell, now the Director of the Fitzwilliam Museum, Cambridge, showed him a medieval missal, laboriously illuminated. He found that it fatigued him to look at it, and said that such books ought never to be made. Cockerell replied that such books relieved the tedium of divine service, on which Butler made a note ending thus:
Give me rather a robin or a peripatetic cat like the one whose loss the parishioners of St. Clement Danes are still deploring. When I was at school at Allesley the boy who knelt opposite me at morning prayers, with his face not more than a yard away from mine, used to blow pretty little bubbles with his saliva which he would send sailing off the tip of his tongue like miniature soap bubbles; they very soon broke, but they had a career of a foot or two. I never saw anyone else able to get saliva bubbles right away from him and, though I have endeavoured for some fifty years and more to acquire the art, I never yet could start the bubble off my tongue without its bursting. Now things like this really do relieve the tedium of church, but no missal that I have ever seen will do anything except increase it.
In 1848 he left Allesley and went to Shrewsbury under the Rev. B. H. Kennedy. Many of the recollections of his school life at Shrewsbury are reproduced for the school life of Ernest Pontifex at Roughborough in The Way of All Flesh, Dr. Skinner being Dr. Kennedy.
During these years he first heard the music of Handel; it went straight to his heart and satisfied a longing which the music of other composers had only awakened and intensified. He became as one of the listening brethren who stood around “when Jubal struck the chorded shell” in the Song for Saint Cecilia’s Day: