Barkla was born in Widnes and studied at the Liverpool Institute and Liverpool University. In 1899 he went to Trinity College, Cambridge as an Exhibition Scholar to work in the Cavendish Laboratory under J J Thomson. At the end of eighteen months his love of music led him to migrate to King's College, Cambridge where he joined the chapel choir. He gained a BA in 1903 and a MA in 1907.
In 1913, after having worked at the universities of Cambridge, Liverpool and King's College London he was appointed professor of natural philosophy at the University of Edinburgh, a position he held until his death. He married Mary Esther Cowell in 1907.
He evolved the laws of X-ray scattering and the laws governing the transmission of X-rays through matter and excitation of secondary rays. For his discovery of the characteristic X-rays of elements, he received the 1917 Nobel Prize in Physics. He was awarded the Royal Society's Hughes Medal that same year.
The lunar crater Barkla was named in his honour and a commemorative plaque is in the vicinity of the Canongate, near the Faculty of Education Buildings, University of Edinburgh.