Yes and no, but mostly No.
3 U.S.C. § 302 states
The authority conferred by this chapter shall apply to any function vested in the President by law if such law does not affirmatively prohibit delegation of the performance of such function as herein provided for, or specifically designate the officer or officers to whom it may be delegated. This chapter shall not be deemed to limit or derogate from any existing or inherent right of the President to delegate the performance of functions vested in him by law, and nothing herein shall be deemed to require express authorization in any case in which such an official would be presumed in law to have acted by authority or direction of the President.
The bolded section (emphasis mine) roughly translates as "You're presumed to have the authority to carry out your job." However, this leaves the question of "so what is your job?"
There are a lot of executive orders (see notes on § 301) which explicitly designate certain functions. The President has the power to repeal those at any time, but until he does, those functions are included in the various position's authority. So why can't he just repeal them all?
The general duties of each Cabinet position are laid out in the laws that create it. For example:
Secretary of State - 22 U.S.C. § 2651a (a)(3)
(A) Notwithstanding any other provision of law and except as provided in this section, the Secretary shall have and exercise any authority vested by law in any office or official of the Department of State. The Secretary shall administer, coordinate, and direct the Foreign Service of the United States and the personnel of the Department of State, except where authority is inherent in or vested in the President.
(B) (i) The Secretary shall not have the authority of the Inspector General or the Chief Financial Officer.
(ii) The Secretary shall not have any authority given expressly to diplomatic or consular officers.
Secretary of Housing and Urban Development - 42 U.S.C. § 3532 (b)
The Secretary shall, among his responsibilities ... provide technical assistance and information, including a clearinghouse service to aid State, county, town, village, or other local governments in developing solutions to community and metropolitan development problems; consult and cooperate with State Governors and State agencies... with respect to Federal and State programs for assisting communities in developing solutions to community and metropolitan development problems... and conduct continuing comprehensive studies, and make available findings, with respect to the problems of housing and urban development.
Secretary of Education - 20 U.S.C. § 3411
There is established an executive department to be known as the Department of Education. The Department shall be administered, in accordance with the provisions of this chapter, under the supervision and direction of a Secretary of Education. The Secretary shall be appointed by the President, by and with the advice and consent of the Senate.
This shows two different ways of granting the Secretary inherent power which the President does not have authority to remove: Explicitly, where there's a whole long list of powers and duties, and Implicitly, where there's a general directive to manage something, but the specifics depend on other laws or are left up to the President/Secretary. But in either case, by going back to the first quote, anything which the Secretary does in accordance with those goals does not have to be authorized by the president.
So where does the "Yes" come in?
My impression is that a lot of the power and duties of the various Secretaries come from those executive orders, and those can be stripped away. It's not totally shutting down the position, but it'd be a significant step in reducing their authority.
Additionally, I could put forward a case that if the President issues an executive order specifically prohibiting a Secretary from doing something, that might negate the default presumption that the Secretary has authority to do it. It's possible that doing so could end up before the Supreme Court for interpretation, at which point all bets are off.
Alternatively, if the President really wants to make an end-run around the "advice and consent" position, all he needs to do is keep nominating candidates that will clearly not be confirmed. That leaves day-to-day operations in whoever the acting secretary is - possibly someone more acceptable to the President. I don't know if all second-in-command positions are "advice and consent", but if not, that's another way to get around it. Finally, I expect that the laws concerning who takes over if the office of Secretary and their backup are both vacant (which could happen naturally due to a death during a long confirmation process) are vague, so it's possible that the duties would be taken over by a non-consent position and/or a committee (At this point, I'm just speculating wildly)
