How does a company keep programmers from seeing encrypted data?

I work for a company that maintains sensitive data for a government agency. We have what I regard as a very weak encryption system.

Suppose that we upgraded to a public key system, which should be all that we need, since the only thing that we encrypt is passwords. Assume that programmers work on a development system with a database where the sensitive data is fictitious, allowing them to know the private key for the development system without gaining access to actual sensitive data.

After code is transferred to the production system, what is the specific mechanism by which users could enter passwords and access their data without anybody else, with the possible exception of a trusted administrator, knowing the private decryption key for the production system? Does this require the use of a third party?