Huge Tree Causing Problems?

No, they can't sue you. It's your tree and you like it. Just tell him you like the tree and you have no intentions of cutting it down. Maybe if you want to comprimise you could just have it trimmed. If he doesn't like the leaves he can move. Get himself a leaf blower. HGe wouldn't paint his house bc you didn't like the color would he?