Throughout her storied life, Marianne Faithfull has been in a tussle with her reputation. Sometimes it looks like a dance. Often, it is more like a fight. Though she’s now revered as an elder stateswoman and a valued collaborator – Warren Ellis is her latest pet – this compilation explores the first two acts of Faithfull’s career.

• ORDER NOW: Björk is on the cover of the latest issue of Uncut

In the clichéd telling of it, Faithfull was manipulated and underestimated, if not exploited, during her pop career, before clambering from the wreckage and finding her own voice. If Faithfull sounded damaged – vocally, she did – the point was underlined. She has dismissed her early recordings as “cheesecake”, though her habitual flintiness has prompted others to diminish her achievements too: in a famously combative interview with Lynn Barber, the journalist tried to extract some small revenge by suggesting that Faithfull was “a singer with one good album”. In which case, why does she continue to fascinate?

The good album in Barber’s reckoning is Broken English. While it’s true that the 1979 LP marked a clear kink in the road and is widely considered to be Faithfull’s masterpiece, it now sounds like a time-stamped product of the new wave era. Those squelching synthesisers go in and out of fashion, but they have a whiff of post-punk cosplay, just as Mark Miller Mundy’s production is identifiably from the Island Records colour chart with its understated insinuations of reggae and roots. What makes the record work is the surprising harshness of Faithfull’s voice colouring the proud alienation of the songs.

The broken English thing is Faithfull, but the album’s title track – here in the 1980 single version – has a lyric which reportedly admonishes Ulrike Meinhof of the terrorist Red Army Faction, though the lyric evinces a more general mood of Cold War world-weariness. The guitar sounds like a bear taking a chainsaw to a barbed wire fence. The actual stand-out from Broken English is “Why’d Ya Do It?”, which is a bit reggae, somewhat rock, a lot Grace Jones (though Jones was still in the business of perfecting her brand of Island ice). “Why’d Ya Do It?” remains an extreme song, with a jealous Faithfull snarling through a litany of sexual grievances (cock-sucking, snatch-spitting, cobwebbed fannies). It’s interesting once, but you wouldn’t want to share a bathroom with it.

This two-disc set spans 1965–95. The title is a neat steal from a book of William Blake’s poems. So what of the cheesecake? What if we ignore the prejudices which came from Faithfull being a symbol of dumb beauty – the “angel with big tits” exploited by Andrew Loog Oldham, the girl in Mick Jagger’s rug – and listen to the songs? They are mannered, certainly. Even Faithfull’s innocence has its duality. The chamber pop songs are very 1960s, while the folk tunes are more knowing in their evocation of olden times.

Faithfull is said to prefer the folkier material, and as a performer she’s smart enough to know that heightened innocence can be chilling. “What Have They Done To The Rain?” adds percussive raindrops to Faithfull’s English rose, and – if you spritz on some sulking – contains the raw ingredients of The Jesus And Mary Chain’s entire career.

Faithfull is less convincing with more famous songs. A live version of “Yesterday” recorded for the BBC Saturday Club doesn’t quite catch the full power of the song’s yearning, and a faintly gothic folk arrangement of Ewan MacColl’s “The First Time Ever I Saw Your Face” is closer to Peggy Seeger than Roberta Flack, but remains underpowered.

She has better luck with Donovan and Bert Jansch. A pretty, chaste interpretation of “Sunny Goodge Street” (a previously unreleased take) clears some of the fog from Donovan’s song. “Green Are Your Eyes” (Jansch’s “Courting Blues”) has a chilly simplicity to it. “Love can be broken, though no words are spoken,” she sings, suddenly sounding more than girlish. Then there is “Hier Ou Demain”, a playful collaboration with Serge Gainsbourg, written for the TV comedy musical Anna. It represents a path not taken.

The mood switches abruptly with “Sister Morphine”, which Faithfull mostly wrote (while having to fight for her credit). It’s an extraordinary lyric, sung from a hospital bed with the scream of an ambulance in the narrator’s ear. Just as it signalled a darkening of Faithfull’s perspective, it became a kind of Frankenstein, haunting her with its drug-soaked morbidity.

And so we come to the songs of experience. Looking backwards, the shadowy corners of Faithfull’s songbook define her image. “Where did it go to, my youth?” she croaks on “Truth Bitter Truth”, while on Tom Waits’ “Strange Weather” producer Hal Willner repurposes her half-spoken narrative style into the manners of a Kurt Weill cabaret. We know about “The Ballad Of Lucy Jordan” (a slightly odd electro production) and the cabaret phrasing of “Strange Weather”. But listen to the way Faithfull repoints the Ruben Blades/Lou Reed song “Calm Before The Storm”, diffusing the epic pomposity of Blades’ recording, replacing it with a note of chilly resilience. It’s the Marianne Faithfull thing: stay calm, embrace the storm.