I don't think you muffler would be the problem as long as you bought a true diesel exhaust system from a reputable source.
Where do you live? At what elevation? I can remember one TDR member, K_Arts, claiming a loss of HP after the 4" install but he was at elevation and I think that played a role in it. Also that was a 24v, us 12v guys have a little more bottom end adjustability.
Just out of curiousity, when was the last time you adjusted your valves?