How to package settings file with setup wizard

Yes you can as the RessourceManager provides such abilities, you have to distinguishe dealing with settings (configuration file and configuration manager) and dealing with ressources (resx, ressources files and RessourceManager)

I was hoping there is some way to be able to change the settings on the client, like a .webConfig file in .net.

All settings have to added to a ressource file, then they could be embeded within the application assembly  to generate a ressource file that holds all your customized settings required by the application then use the Resgen.exe tool and then use the compiler csc the C sharp compiler to embed resources directly witihn the application assembly it is the best solution.