If you have new growth showing, then yes - you can certainly take it off. For anything a bit less hardy, or with no new greenery showing yet, it's worth leaving some on for now. I sometimes take part of the dead stuff off, as it can actually be a problem in the wind. It can uproots plants depending on whereabouts they are. Otherwise, I only take it off on the hardy plants that have growth, but it's usually too early at this time of year for that. The mild winter has meant there's a couple of things showing though.
Yes it's time to tidy up perennials before they start growing again as it makes it more difficult when new growth has formed. Some people do it in the autumn and others prefer to do it in the spring. Have you got space in your bin or barrel for the trimmings Ann85? or space to make compost heap?