Westminster was once an island at an important ford across the River, but the name originally referred only to the Abbey.
Once upon a time, the area we now call Westminster was an island, called Thorney Island, formed by the River Tyburn, which split, delta-like, into branches (“distributaries”) as it entered the Thames. Thorney Island was a piece of land between its two main branches. Before the Romans founded London, an ancient route from Sandwich to Chester and Holyhead may possibly have crossed the Thames at Thorney Island, where it seems there was a natural ford. The ford would have been superseded when the Romans built Watling Street (incoporating the Edgware and Old Kent roads) via what’s now the City, where they’d installed London’s first bridge.
From around the eleventh century there was a horse ferry from the end of Horseferry Road (where else?) over to Lambeth Palace. A horse ferry is a ferry on a barge long and wide enough to take a horse and cart. The ferry was the only available crossing here until Westminster Bridge was built in 1750, and it continued in operation until Lambeth Bridge replaced it in 1862. You can find more about the origins of Westminster’s street names here.
Westminster Abbey, according to legend, started life as a monastery, founded at the very beginning of the seventh century by Sebert (or Sæberht), the first king of Essex to adopt Christianity. London was at that time under Essex’s control. It seems that a fisherman by the name of Edric gave a stranger a lift across the river in his boat on the evening before the monastery’s church was due to be consecrated. The stranger turned out to be none other than St Peter, who rewarded Edric with a fine catch of fish. He told Edric to give a salmon from his catch to the Bishop of London and inform him that he didn’t need to consecrate the church as Peter had already done it. And to this day, every year on St Peter’s Day (29 June), London’s Worshipful Company of Fishmongers present the dean of the Abbey with a salmon in memory of the event. Sadly, the fish is no longer caught in the Thames, which hasn’t really had a salmon population since the early nineteenth century.
St Peter’s church became an Abbey around the late tenth century, known as Westminster. There was also an Eastminster, a Cistercian abbey which stood from 1350 until 1538 on Tower Hill, where the mint was later located. Edward the Confessor (king 1042–66) had Westminster Abbey rebuilt but it wasn’t finished until just before he died. His successor Harold Godwinson, the last Anglo-Saxon king of England, was the first to be crowned there, in 1066. The second was William the Conqueror, who ousted Harold. He was crowned in the Abbey on Christmas Day the same year. Today’s building mainly dates from 1245, in the reign of Henry III, but a few bits of Edward the Confessor’s older structure still survive.
The first king to have a palace at Westminster was the wave-defying King Canute (who ruled 1016–35), but Edward the Confessor made it his main residence. The first parliament was held there in 1265, and parliament has met there ever since. Its old building however burnt down in 1834. The new building was designed by local lad Charles Barry, who was born at 2 Bridge Street, which would have been opposite Big Ben if he’d built it before he was born.
Westminster became a city in 1540, at which time there was countryside between it and the City of London. The area between them was built up over the centuries as the old Anglo-Saxon wic – a settlement centred roughly where Covent Garden now is – gradually expanded westwards. As a result, London is arguably not a city, but a big town with two cities inside it. Westminster absorbed Paddington and Marylebone in 1965 to become the City of Westminster that we know today.