Can a client place a bug in my company and use it as evidence?

No she can't do that; but let her try, that will give you grounds to file criminal and civil charges for illegal recording and invasion of privacy (assuming no illegal activity was recorded), and the evidence wont be admisible in court.

Besides she has no evidence of anything since it's 100% legal to discuss, withing the privacy of your company, the late payments of a client that refuse to pay; it's also legal to kick that client.

Just a few year ago private detective Anthony Pellicano was sent to jail for "unlawful wiretapping".

http://en.wikipedia.org/wiki/Anthony_Pellicano

But of course it would be better for you to hire a lawyer if this go to court.

You need to check with a legal mind, since laws vary on taping conversations vary from state to state. She may actually be the one in trouble, if your state's laws prevent recording conversations without permission. For instance in PA, "It is a felony of the third degree to intentionally intercept, endeavor to intercept, or get any other person to intercept any wire, electronic, or oral communication without the consent of all the parties. 18 Pa. Cons. Stat. § 5703(1)."

Any taped evidence in court would depend on too many factors to answer here.

What is this late-paying mom hoping to gain? Act with caution, but also act with knowledge.