Can you donate money to fund part of the government?
Yes, private entities have donated money to fund functions of the federal government. For instance, much of the contents of the Smithsonian Institution are gifts, and the Institution was founded by a trust fund left to the US by a James Smithson. That's one of the earliest and best examples. More recently, in 1979, 41 agencies accepted over $20 million in gifts (GAO). However, these were all pursuant to specific or standing authorization by Congress to accept gifts.
General rules for gifts
As a generic rule, any miscellaneous money collected by an officer or employee of the United States for any reason has to go into the general treasury; this is according to 31 USC 3302, which was enacted to keep some sort of centralized control over government money. Congress can make exceptions; they have done so in several cases. For instance, the Department of State can accept donations for its use, which are automatically appropriated to the Department. The Defense Department can accept donations as well; those are held in trust, but cannot be used without being appropriated. However, most other agencies can't accept such donations, and no agency can use donated money to fund its mission without Congressional approval.
(The one bit of an exception is employee morale/welfare/recreation funds, particularly military ones; these have a long history, and are somewhat a governmental function, and are recognized in law, but weren't created by statute. However, the thing with them is that they're funded directly by the people they benefit [employees], and it's not really what you're asking about).
Why can't they use gifts without approval?
The basic rationale is that Congress's control over the budget is meant to be a tool to give Congress general control over what the government can do. For instance, suppose Congress wants the government to generally back off from extremely strict environmental regulation. They can do so by cutting EPA funding, forcing the EPA to restrict its activities to the most important environmental issues, without having to say what those are (which is the EPA's area of expertise). If the EPA could take money from environmentalist groups, that would severely reduce Congress's control over how intensive US environmental regulation is. A common claim about the ATF from gun control advocates is that it's underfunded in order to handicap federal gun control measures; whether that's a good thing or a bad thing in your opinion, the relative importance of gun control is the sort of policy decision that's supposed to be made by Congress, not by the Brady Campaign.
To that end, the rule of thumb is that an agency can't augment its appropriations with income from any other source. Standing appropriations like State has are fine, but without them, it means that Congress gets to set a cap on funding (even with them, what Congress giveth Congress can taketh away). There's no single law saying that; the main things cited in favor are 31 USC 3302 (the Miscellaneous Receipts law we saw earlier), 31 USC 1301 (which says an appropriation can only be used for the thing it was appropriated for, so no using one department's budget for another department), and Article I, Section 9, Clause 7 of the US Constitution (which says that no money can be spent from the Treasury without an appropriation).
There's also the Antideficiency Act, which says that an agency can't obligate itself to spend more than the sum of any funds it has and what it's been appropriated, which has the same goal: no spending more money than Congress wants you to be able to spend (before it was enacted you'd see things like the military "accidentally" running out of money a couple months into the year, so sorry, our mistake, of course we can't defend the country now; oh, "have some more money"? gee, what a surprise, thanks!). That law is a) what actually causes shutdowns in the first place (employees can't work for IOUs because there's no appropriation; they can't volunteer their labor, because the act also generally bans that), and b) means that a donation which isn't automatically appropriated can't be used for anything until it's appropriated (an agency can't say "well, we'll get the money as soon as it's approved"; they have to wait for appropriation). The money can't be used for anything else, as far as I can tell (conditional gifts must be used according to the conditions); it seems to just sit around until Congress approves its appropriation.
Note on payroll
You mentioned payroll as something you're not asking about because other branches would be involved, but it's actually not allowed for other reasons: 18 USC 209 makes it generally a federal crime to be paid by a third party for doing your (paid) job for the federal government. The reasons should be fairly obvious (i.e. to counter corruption). That's another major concern in general for a private organization funding a government agency: the risk of the government agency working for that organization.
Legislative stuff I mostly cited, much of the rest is from here (start at page 236).