What is the best way to Koh Samet to Rayong & back?

Where are you starting from Jeff. That would help in answering your question. But what hell let just say take a taxi is of course the best way to do anything. Yes it's very easy to caught a boat to the Koh Samet, but to go their to just use the beach and not stay is stupid. Koh Samet is considered a National Park and they will charge you once you arrive. I forget the price something like 300 baht for each foreigner. They do have a beach on Rayong but I hear it's also pretty polluted like Pattaya. I recommend you stay on Koh Samet for a few days if you go. But again your question is so poorly written that maybe you can edit it so we really know what the hell your asking.

There is a sort of soft charm to Rayong along the coastal areas, but one look into the sea at all the refineries and one look into the news about the incredibly high incidence of cancer in the areas nearby - sort of blunts the charm for me….

And yes, the best way would be to take a taxi to the ferry to Samet… I am not sure I understand why you wouldn't be able to use the beaches - I assume the oil spill issue is cleared up by now.