Don't remove it!

The minter contract is the parent and wallet is the child. There is one minter instance and N wallet instances (N is the number of holders of your token). Getting rid of the wallet would mean that your users wouldn't own or be able to do anything with your jettons.