He was born at Calais and trained at Antwerp (under Wappers), in Paris, and at Rome, where he came into contact with the Nazarenes. Settling in England in 1846, he became a friend of the Pre-Raphaelites and--with his taste for literary subjects and meticulous handling--an influence on their work, though he was never a member of the Brotherhood. Rossetti studied briefly with him in 1848 and Brown's Chaucer at the Court of Edward III (Art Gallery of New South Wales, Sydney, 1851) contains portraits of several of the Brotherhood.
His best-known picture, The Last of England (City Art Gallery, Birmingham, 1855) was inspired by the departure of Woolner, the Pre-Raphaelite sculptor, for Australia. The other famous anthology piece that Brown painted, Work (Manchester City Art Gallery, 1852-63), shows his dedicated craftsmanship and brilliant coloring, but is somewhat swamped by its social idealism. In 1861 Brown was a founder member of William Morris's company, for which he designed stained glass and furniture. The major work of the later part of his career is a cycle of paintings (1878-93) in Manchester Town Hall on the history of the city. Brown was an individualist and a man of prickly temperament; he opposed the Royal Academy and was a pioneer of the one-man show.