I've had my silencer ring out for a couple years. After taking it out, I couldn't tell any difference other than the sound. Also, it's easy to put back in if you don't like it.
With my PSM modded stock airbox running an AFE PG7 filter and no silencer ring, the turbo is fairly loud. Same setup but substitute HD paper filter for AFE and it really quiets down.