A fundamental principle of business is that you do things in house that you think can give you a competitive advantage and outsource things that you don’t. At an early-stage technology company this means you do in house: product design, software and/or hardware development, PR, recruiting, and customer relations/community management. Ideally, most of these activities are led by founders. You should outsource legal, accounting, website hosting, website analytics etc. (Unless you are starting a company where one of those activities can give you a competitive advantage, e.g. a securities trading startup would need to have in-house legal).
A lot of startups over outsource. A few years ago, you’d sometimes hear tech startups say they were going to outsource software development. Thankfully, founders have gotten smart about this and it rarely ever happens except as a stopgap. It is still common for startups to hire outside PR firms. If you decide to hire an outside PR firm, that means you don’t care about PR. Just because you are willing to spend some of money on it doesn’t mean you think it’s important. You probably shouldn’t hire an investment banker during an acquisition unless your company is later stage. And you might occasionally use an outside recruiter but the core recruiting activity needs be done by founders.