It was designed by William Williams in 1812 in accordance with the utilitarian principles laid down by Jeremy Bentham. The Williams design was subsequently adapted by Thomas Hardwick who began the construction. Hardwick resigned soon after and John Harvey took up the role. He was dismissed in 1815 and Robert Smirke took over and completed the project. The first prisoners (females) were admitted on 26 June 1816 and the first males in January 1817. By the end of 1817 the prison held 103 males and 109 females with 452 males and 326 females in late 1822. Sentences were from five to ten years and in the penitentiary's early years were given as an alternative to transportation to those thought most likely to reform. Later it ceased to have a penitentiary function and became a holding centre for those awaiting transportation or in the case of infirm prisoners, removal to the hulks.
In the Handbook of London in 1850 it was described as
MILLBANK PRISON. A mass of brickwork equal to a fortress, on the left bank of the Thames, close to Vauxhall Bridge; erected on ground bought in 1799 of the Marquis of Salisbury, and established pursuant to 52 Geo. III., c.44, passed Aug 20th, 1812. It was designed by Jeremy Bentham, to whom the fee-simple of the ground was conveyed, and is said to have cost the enormous sum of half a million sterling. The external walls form an irregular octagon, and enclose upwards of sixteen acres of land. Its ground-plan resembles a wheel, the governor's house occupying a circle in the centre, from which radiate six piles of building, terminating externally in towers. The ground on which it stands is raised but little above the river, and was at one time considered unhealthy. It was first named "The Penitentiary," or "Penitentiary House for London and Middlesex," and was called "The Millbank Prison" pursuant to 6 & 7 of Victoria, c.26. It is the largest prison in London. Every male and female convict sentenced to transportation in Great Britain is sent to Millbank previous to the sentence being executed. Here they remain about three months under the close inspection of the three inspectors of the prison, at the end of which time the inspectors report to the Home Secretary, and recommend the place of transportation. The number of persons in Great Britain and Ireland condemned to transportation every year amounts to about 4000. So far the accommodation of the prison permits, the separate system is adopted. Admission to inspect - order from the Secretary for the Home Department, or the Inspector of Prisons.
A single buttress remains by the river with the inscription Near this site stood Millbank Prison which was opened in 1816 and closed in 1890. This buttress stood at the head of the river steps from which, until 1867, prisoners sentenced to transportation embarked on their journey to Australia. The prison boundary can still be seen in the building and road layout of the area.
The prison is evocatively described in Sarah Waters' 1999 novel Affinity.