The answer by JamesK is right as far as it goes and I agree with it, but this answer spells out some of the operative principles with a little more detail.
The President, and the executive branch more generally, in U.S. Constitutional law, is charged with carrying out and "executing" the laws of the United States, including its treaties and the U.S. Constitution.
The President may do so in any manner that is consistent with those laws, but has wide discretion in determining precisely how to enforce the laws.
An Executive Order is simply a Presidential level determination made by the President about how the laws of the United States will be enforced.
For example, while the FBI has the legal authority to investigate all cases of mail and wire fraud, it doesn't have the budget to prosecute every single case that it learns about. So the FBI, using authority derivative of the President's executive branch authority, has adopted a legally non-binding, but fairly strictly enforced official policy that it will generally only prosecute mail and wire fraud cases where the harm done exceeds $75,000, while leaving other mail and wire fraud cases to civil lawsuits and state court systems.
Presidents are entitled to make policies telling federal civil servants how to carry out their jobs in most circumstances, because perfection isn't possible and often isn't even well-defined.
Some matters are expressly delegated to the President to make by executive order (e.g. national disaster determinations). Some executive orders implement binding court decisions (e.g. executive orders that direct the federal government to treat racially restrictive real estate covenants as void). Some executive orders simply implement a policy of a Presidential administration that is within a subject matter area where there are federal laws that exist that the President has some authority to enforce.
If Congress doesn't like a determination made in an executive order, one of the most common ways it responds is by inserting a rider in an appropriations bill related to that executive order, since they President can only used federal funds to carry out policies when there is a relevant Congressionally authorized appropriation of funds to do so.
What determines could be better handled through legislation and on the basis of what?
One question is whether the authorization from existing statutes is clear enough.
Another principle of U.S. Constitutional law, called the non-delegation doctrine, provides that there are some decisions which Congress simply cannot delegate to the executive branch. The U.S. non-delegation doctrine is explained at the link above as follows:
The Supreme Court ruled in J. W. Hampton, Jr. & Co. v. United States
(1928) that congressional delegation of legislative authority is an
implied power of Congress that is constitutional so long as Congress
provides an "intelligible principle" to guide the executive branch:
"'In determining what Congress may do in seeking assistance from
another branch, the extent and character of that assistance must be
fixed according to common sense and the inherent necessities of the
government co-ordination.' So long as Congress 'shall lay down by
legislative act an intelligible principle to which the person or body
authorized to [exercise the delegated authority] is directed to
conform, such legislative action is not a forbidden delegation of
legislative power.'"
Very recently, the U.S. Supreme Court has also restricted this authority by creating out of whole cloth what it has called the "major questions doctrine" which provides that: "if an agency seeks to decide an issue of major national significance, its action must be supported by clear congressional authorization."
Incidentally, parallel separation of powers doctrines in U.S. Constitutional law also prohibit some kinds of Congressional regulation of Presidential discretion, particularly in matters of foreign affairs.
What is to stop a president from establishing a new
government body like the EPA on the basis of an executive order?
In addition to the necessity of some sort of legislative authorization that is sufficient to cover the proposed governmental body and an appropriations bill sufficient to fund the agency, as discussed above, most (but not all) executive orders must confirm to the Administrative Procedure Act (APA).
The part of the APA relevant to the authority of the President to issue Executive Orders specifies what executive branch determinations count as "regulations" and what procedures must be followed to adopt or amend a federal regulation.
Under the APA, for example, an executive order authorizing U.S. military forces to retaliate against an attack by Iranian based forces in Syria (an order that President Biden recently issued) would not require that the APA process be followed and is directly authorized by the U.S. Constitutional provision that the President is commander-in-chief of the U.S. military, while a modification of the rules for taxing international sales of goods between related parties with transfer pricing, in contrast, would clearly be governed by the APA process.
The line between which executive orders count as regulations and which ones don't is a somewhat arcane and technical area of the law, but certainly, creation of a new permanent government agency would typically require compliance with the APA process (although a short term ad hoc task force might not).
