The idea of a museum for Wantage began in 1958 when the Urban District Council made an appeal for objects to form a town collection. The response was so positive that in the 1960s the growing collection was housed in the Civic Hall, and then the Victoria Cross Gallery in Wantage Market Place.

In 1972 the Vale & Downland Museum Trust was formed. The Trust persuaded the Vale of White Horse District Council, newly formed in 1974, to buy the Old Surgery building and lease it to the trust. Initial conversion works were carried out and Museum moved into the Old Surgery in 1975.