Job offers after just starting a new job!??!?

If the pay and benefits are better, there's nothing wrong with moving to another position. You don't have to apologize for that. You must do what is best for you and your family and situation. 

However--consider what working for the Post Office actually means before you move into that job. The hours are long. The working conditions may be harsh. A mail handler is an inside job--but it means standing for long periods of time, carrying heavy boxes or packages, doing sometimes very repetitive and boring work. And what shift did they give you?  Some shifts in some areas are late at night, or all night, and may not be convenient. 

You owe it to yourself to examine the job a bit more closely--and if it is indeed better, then go ahead and take it--and don't look back. But if it has drawbacks, maybe staying where you are is a better plan. Postal workers DO make decent money and have good benefits--would that sway you? It's also a job that has pretty solid staying power and job security. Just look into it more and then make a decision. 

It's common to find a better job and give notice to your current employer. You have to do what's in your best interest. I would, however, compare the retirement, salary, and other benefits and decide on the one that I feel is best for me if I were you. The post office has a reputation as not being a great place to work and their financial situation is dire.

USPS is notorious for having disgruntled employees. 

It would be fine to leave your current job. That employer would treat you the same way if they wanted to. 

When I hear of an employer that treats its workers well I think of it as a fantasy or a myth. Truly I don’t believe there is such a thing.

If the other job is much better, go ahead and take it.  good luck.

You aren't an indentured servant. You are free to resign your current position.  It will leave a negative impression with your current employer, so don't expect to use them as a future reference.  

As other's have noted, you probably want to make sure that the job duties, hours, and compensation package are "better" before jumping at the offer. 

Are you sure the postal job is better than where you are?  Check Glassdoor for employee reviews. There are a lot of disgruntled employees due to the way the service is run now. Everything is timed and you're constantly told to move faster.  Government jobs can be great but they can also be very stressful.

Did you get a job offer or an interview with USPS?

USPS often moves very slowly during the recruitment process.  If this is the first time you have heard from USPS since submitting the application, it could be months or even more than a year before you will be hired.

Which job do you want more? Which one pays better, which one has better future potential, which one are you overall more interested in? 

Does it look good to quit a job very shortly after starting it to take a different job? No, but you would be far from the first person to ever do that. If you want the USPS job more, then take it. Just know the company you are resigning from should never be reapplied to or used for a reference in any way.

You'll make no friend there obviously.  You would be stupid to give up a much better job though if that is the true path you want with you career.

No, it's your life and you will be replaced in a week.