Are aphids a danger to young trees?

1108923

I've a Wild Cherry and a Rowan - the Cherry is about 1 metre tall, the Rowan half that - and I've noticed the ants in my garden are cultivating aphids on them. Is this something I should be worried about and treat now while there's only a handful, or is it fine to leave them at it?

I'm going to be planting out some tomatoes soon as well, so will having two aphid harbouring trees nearby make the tomatoes more likely to be infested too?

Thanks in advance