When it comes to performance, less is more. Always delete plugins you aren't using (however, the number isn't as important as quality) and keep your custom CSS to a minimum. Every little bit helps!