Japanese maple health

irinag

Hi, I looking for some advice. I have an established Japanese maple in my garden. I noticed last summer that the leaves were falling off prematurely on part of the tree (the trunk on the left in the attached photos), and now the leaves are not returning on the same trunk of the tree. There had been some fungus on that trunk for a while (seems to be gone now). Is the tree dying, or will it be okay if I get that trunk removed? (or will it just topple over ?!?) Is there anything else I can do?

Many thanks in advance for any help!