What am I missing in choosing yarn the correct weight for crochet?

I know EXACTLY what you mean. I've had sock yarn that seems closer to #3 than to #1, and I've bought #4 (Worsted) weight yarn and it appears to be an obviously lighter gauge than what is needed for my patterns! This is especially noticeable if you're going from a fiber like wool to a lighter fiber such as bamboo. The worsted weights on them will not work the same.

What I try to do is determine the fiber content of the original work (wool, cotton, bamboo, acrylic etc.) and match that up with whatever yarn I'm using.

There are some acrylics that have a smoother texture (satin by Bernat & similar) and they don't have the body. In that case, try and determine what texture and body you need and match it up. (a worsted wool would equate with a rougher textured acrylic of the same weight.)

If you can, as people have noted, knit or crochet a swatch, not just for size, but for body and texture. If you don't like it in 4x4, you won't like it when it's done.

I'm afraid that I might not be as clear as you need. Just use the numbers as a guide, not as your end destination!

This is rather complicated to answer because there are many factors that go into making any one particular yarn successful in a project. Weight or size of the yarn is only one of them. The way it was spun is a major one for me. The fibre content is another -- acrylics just do not behave the same way wool will, which is different than cotton, which is different again from silk.

The first thing in the yarn store is to compare to see if it is the same size. You need a sample of one yarn, and then compare it to another. The best way is to cross the lengths of two yarns and then fold them back on themselves. You will get them hooked on each other (like you link your fingers together) and then you can clearly feel any differences. You can sometimes see it too, but I find I can feel the difference as I run my fingers over that strand back and forth.

When you think you have a yarn that may work, you still need to work up a sample swatch. Sorry, it is a pain, I know, but you won't know until you actually use it to see how it will behave. Some yarns compress too much, some stretch, some have no body.

I never look at the labels of yarns because I spin my own. But as a general rule, I use the crochet hook as a guide. The hook is about the same thickness as two strands of yarn held together. That is a general guide. If you want a tighter fabric, use a smaller hook. If you want a looser fabric, then use a larger hook. That is about the easiest way I can describe it. If you can't find the yarn you want, then change the size of the hook.

The only correct way to find the substitute for a yarn is going by length per weight. In the US, that is yards per pound. In the rest of the world, that is metres per kilogram. But remember: that is for the same fibre content. Silk weighs more than wool, so you can't compare different fibres. Worseted spun is different than woolen spun.

I am having the same problem as you and I hesitate to write anything here, because I am no authority. So that's the warning. I look at the yarn they suggest and see how thick/thin it is (the number too)...and then I choose something similar. I also buy more than I need and figure that if I have a skein left over and it's not too expensive then I will just make a hat and donate it for charity. So far it's working well for me. I do know that the person who wrote about gauge is correct. You have to make a swatch to see if what you are doing matches the intent of the pattern. But I know that's not what you are asking. I also shop at yarn shops now, because they are the authorities and they help me with what will work and why. I've been learning.

You need to make a 4x4 in square to see how many st in an inch.

It usually will state 20 st =4in for whatever the yarn is.

This is called guage.

Here is a link about guage explanation.