When restoring A/C operation what comes first?

OK so my A/C system is intact but has never worked since I bought the car. I'm finally getting off my butt and fixing it. The original hoses were cracked and the freon leaked out. I've replaced the hoses but I've read the receiver dryer should also be replaced. That's not a problem but when you suck out the old oil (or whatever is left in there) won't it contaminate the new receiver dryer? Am I correct in thinking the system should be drained first and then have the new receiver dryer put on?