What is the green fungi that is on the bark and branches of our trees?

Many people mistake that green stuff growing on tree trunks as a fungus, but it IS NOT! It is called a lichen! It does NO harm what so ever to your trees. Spraying your tree with a fungicide will do nothing to the lichen (because it is not a fungus).

Now, those little things on the leaves that look like little worms, that is an insect that is laying eggs in the leaves of the plant. I have seen this before.. if you try to pick one off, you end up pulling the leaf away with it. Here's a picture: http://hflp.sdstate.edu/IMAGES/HACKGALL.JPG

Sounds like you have a lot going on. You should probably contact your local extension service. They can help you with soil tests (which you should have done), they can tell you if you need to fertilize and what you can do with those little wormy things on your leaves.

Sounds like your tree needs a little TLC! Without knowing where you are and the tree type this is the best info I can give you.

If the green stuff grows only on the North side of the tree , that is moss. The worms are a separate problem

moss appears to be the fungus . but more worrisome is the little caterpillars that are munching away merrily ; try making a mixture of garlic ( not too strong ) water and white oil (again not too strong ) in a spray bottle ; and give that a go , remembering to do under the leaves as well the top , and indeed the the branches .

The green stuff isn't fungus, it's algae. It probably isn't damaging your tree, but the worms may be some kind of larvae that are eating the leaves. Insecticides will probably poison your fruit, but there are natural insecticides that you can make at home with Listerine, dish soap, garlic and tobacco juice.

It's the same stuff that is growing on your teeth from not brushing.

They are called LICHENs

they do not normally kill trees.