how do we know what stuff in space is made out off?

They use a Spectrometer.

https://en.wikipedia.org/wiki/Spectrometer

We analyze the light we see it with. If the chemicals and atoms in a nebular are excited by a nearby star - meaning, if the light of that star forces electrons into higher orbits - then, when the electrons fall to their nominal states, they emit photons in certain wavelengths... Hydrogen emits light in two wavelengths, water in 3, oxygen in a single one that I know of, and alcohol has a few also. We analyze the light we see - and can determine what chemicals are present by the "fingerprint" in the spectrum... Starlight shining *through* a cloud will have some of it's light absorbed, leaving black gaps in the rainbow spectrum we expect (and absorption spectrum), and other times there's small bands of light in different colors against a black background - this is an emission spectrum.

Spectral analysis. We analyze the light bouncing off it, and compare it's signature to known chemicals.

Spectroscopy mostly.

Space probes