At what point have I unsulated a wall enough where I start to get less heat loss.?

Assuming you have insulated the walls, floor and ceiling, your windows and doors are probably the culprit. Single pain glass is useless, double pain glass is fair, triple pain glass is recommend. The best insulation requires staggered studs so that no stud contacts both sides of the wall. Be sure your walls have a poly vapor barrier and all electrical outlets are sealed. Heat is like water in a bucket: there is no point adding more insulation to one side if there is a big hole elsewhere..

to properly  insulate you have to  cut out  air flow  through walls  and recepticles and  aother oapenings  too..  you will probably not be too far..  but  there is another consideration  as you  seal the house  to heat loss you must  make sure aire flows in through  heaters or other vents to allow   air to be  exchanged

You just fill in between the studs with what ever thickness matches the stud size. An old house is probably 3.5 inches. of thickness. Many newer homes or additions to older ones can be 5.5 inches. 

What amount is optimum is a complex calculation since the place where you live, the cost of energy and even the square footage of walls or ceiling has an impact on the optimum amount of insulation. The type of insulation and its cost is the other variable. 

Heat loss starts going down with the first layer of insulation.  the amount that every extra layer reduces is less and less.

You are overthinking it.

Any insulation helps, there is no cutoff point as you imply. The relationship is very linear. Two inches of insulation provides half the heat loss as one inch, for example. 4 inches 1/4 the loss of one inch. or, 4 inches of R40 provides half the heat loss as 4 inches of R20, to make up numbers. As others said, use the insulation rating (or higher) required in your area. Be sure to seal around the edges, outlets, pipes, etc. 

INSTALL AT LEAST ONE FOOT OR MORE THICKNESS FIBER GLASS TO COVER THE WHOLE WALL. AFTER COMPLETION, TEST THE WALL TEMPERATURE WITH A LASER BEAM TYPE TEMPERATURE MEASUREMENT TOOL TO MAKE SURE THERE IS NO HEAT LEAKAGE ALONG THE WALL,JOINTS AND EDGES.

This is dependent on where you live and the temperatures, Hi and low, in your area.

You may want to ask at your city building department. Too much is a waste of time and money, too little won't pay back sufficiently.

Make sure you include a moisture barrier.

Don't commit anything to the building department. They may want a building permit. Just tell them you are researching for an article in a paper or book.

To answer the basic question, any amount of insulation reduces heat loss.

There is a basic point where more is uneconomical.

R19 is standard for walls - code. R40 is standard for ceilings. If you don't want to lose square footage in your home, don't add to it. Just use the R19. That's standard in the desert so it is probably code in the north too. Check the code where you live and use that, just in case it is more than R19. You should really put in what's code in your area.

The insulation starts working, a little bit, as soon as you put some on. It's not like putting on a bit makes things worse. But the effect is pretty small until you get to where 10% - 20% is covered. Even then the effect is small. You need insulation on at least 50% of the outside walls and ceiling, to make it really worthwhile.