Native Spanish speakers. Do you use "se me" in your conversation?

Yes, that's very frequent and normal in Spanish. It is a very Spanish structure.

First, verbs like "perderse", "caerse", are used mostly as reflexives by itself; so, you never say:

"las llaves perdieron", but "las llaves SE perdieron" (the keys got lost)

or

"la copa cayó" but "la copa SE cayó" (the glass fell).

When you use one of those verbs, the meaning is that the action has no agent (nobody lost the keys; nobody made the glass fall). In the case of "caer", you can say "la copa cayó", but you need to add a complement indicating where the glass fell: "la copa cayó al piso" (the glass fell to the floor), and even in that case, it is very usual to say "SE cayó".

Adding ME, TE, NOS, etc., indicates that the action had an agent (somebody lost the keys; somebody made the glass fall).

So:

Se ME perdieron las llaves = I lost the keys

La copa se ME cayó = I let the glass fall / I drop the glass

Se Te perdieron las llaves = You lost the keys

Se TE rompió la camisa = your shirt got tear off.

Yeah, we use it. Se doesn't only work as reflexive, there's a lot of different types of ses (my spanish teacher told us that when she was doing the career they have a subject called: "usages of se"), but in the examples you've written you have to say se because all the verbs are pronominal verbs (caerse, perderse, olvidarse) that means they're always conjugated with a pronoun.

Yes it is used a lot. I'm a native speaker, so I couldn't give you a good explanation why, but it just has to be used.