I want to make a hole in the side of a glass bottle.?

Try a drill made to drill ceramic tile. I got one free as a sample at a tile supply store. Not a tile store, but a place that sell supplies to tile setters. The blade when purchases costs $2. Try it on an unimportant bottle.

What size hole are you needing to make? If it is fairly small then the best approach will be drilling. Go to your local hardware store and get a bit for drilling ceramic tile. You can get these for just a few bucks. Fill the bottle with cool water and put the cap on or cork it. This will help the glass dissipate the heat from the drill so it will be less likely to crack. Lay the bottle in a container full of sand if possible so it won't role while you're trying to drill.

You want simple and cheap? Get a bunch of bottles and keep cracking them over your head until you get the right size hole.

A diamond hole saw is pretty much the only way.

Makin' a hot-knife bottle eh? O.K. Put a dime in a bottle, spin the bottle till the dime is zingin around and then lightly rap it on your heel...a dime sized hole will result! Too hard and you'll have to start again...