What is the meaning of round bubble, rectangular bubble, & caption bubble in a speech balloon/bubble?

I think you're looking for hard and fast rules that don't exist.  The lettering and bubble styles are pretty subjective.  Rectangular bubbles might for example mean that a robot is talking in some kind of monotone or a voice that sounds artificial.  The lettering will be blocky or reminiscent of a digital quality from a screen.  The rectangle meant to show robotic order.  But here's the thing - not always.  I've seen rectangular bubbles in other situations.  

Likewise, a bubble with a very erratic or wavy edge might be to project a feeling or weakness or unevenness to the voice.  Maybe somebody struggling from injury or lacking confidence.  A dashed line on a bubble edge implies a whisper.  A cloudy bubble is a thought bubble.  

Bubbles for a monster that was half speaking half growling my have harsh edges.  Or someone speaking particularly loudly or commandingly we might get a double edge with color accents.

But ultimately, all of this is up to the particular comic or artist doing the art and lettering.  There's no manual AFAIK.  It's about asking what kind of feeling should the reader be feeling from this person's dialogue and how can I visually simulate that in a speech bubble.