I have a stock to 4" downpipe that feeds into 5" exhaust from just in front of the muffler, back. Most of the noise is from the exhaust clanging against my shock and fuel tank, but your truck probably wouldn't have such problems. My muffler is very loud. The more fuel i give it, the louder it gets.
Before i lost my digital SPL meter, i took some readings. My truck was painfully loud. Cruise at the lowest around 90dB and during acceleration would approach 110dB. Even at idle it was around 84dB in the cab. Way too loud! All of these numbers are above levels that quickly cause permanent hearing loss.
I ripped out the entire interior of my truck this summer and installed sound dampening. I can now hear my turn signals until i'm driving up over 75 mph. If i were you, i'd shop around for the quitest muffler that can be found. Possibly even a resonator before the muffler. Many good posts here about the Donaldson mufflers, i think i remember seeing something in a magazine about a hush or quiet diesel muffler from another company, perhaps Flowmaster.
Also look at what products Advanced Thermal Products has on their site, atpwrap.com. My truck is running the diaper on the oil pan and the cover over the valve covers. Everyone asks what those big cloth things are for. They are effective at dampening engine noise. They have kits that will cover the exhaust manifold, turbo, and maybe even the downpipe and muffler. They might be able to custom make something for an aftermarket turbo or exhaust as well.
If your engine needs the 4" exhaust, do it. If it is too loud, consider sound dampening, which you will probably appreciate anyway.