What you're describing sounds like a form of liquid democracy - described by Blum & Zuber (2016)1 as:
a procedure for collective decision-making that combines direct
democratic participation with a flexible account of representation.
Its basic model consists of four components that can be stated as
follows: All members of a political community that satisfy a set of
reasonable participatory criteria (adulthood, baseline rationality)
are entitled to:
- directly vote on all policy issues (direct
democratic component);
- delegate their votes to a representative
to vote on their behalf on (1) a singular policy issue, or (2) all
policy issues in one or more policy areas, or (3) all policy issues in
all policy areas (flexible delegation component);
- delegate those
votes they have received via delegation to another representative
(meta-delegation component);
- terminate the delegation of their
votes at any time (instant recall component).
Their paper mentions that this model was initially proposed in 1969 by James C. Miller, in his paper "A program for direct and proxy voting in the legislative process".2, where he describes a system which sounds very similar to the one you describe:
Another characteristic of our legislative system is that the elected
representative votes on behalf of all people, in his district at all
times, regardless of his degree of competence in the various matters
of national concern. But this need not be the case: instead of
electing representatives periodically for a tenure of two years or
more, why not allow citizens to vote directly or delegate proxy to
someone else for as long as they like (which is, of course, analogous
to stockholder voting schemes in large corporations). Actually, there
would be a wide range of alternatives available. The most concerned
voter would vote on every issue at his personal console. Another may
delegate proxy to someone he feels would vote as he would if only he
had the time and knowledge to participate directly. Most voters,
however, would utilize some combination of these extremes, voting on
major issues personally and delegating proxy to someone else for the
minor decisions. Thus, the third feature of the proposal is a
provision for proxy as well as direct voting.
He even gives an example similar to the one you suggest, where a voter may delegate their vote to a policy expert depending on their concerns:
Under the proposed legislative scheme, voters need not be experts in
each field; neither should representatives. The way is open for the
individual voter to give his proxy to a military expert whom he trusts
to cast a vote as he would if only he had the required military
expertise. At another time, the voter might delegate his proxy to an
economist to decide upon issues affecting employment and inflation.
1: Blum and Zuber. "Liquid democracy: Potentials, problems, and perspectives." Journal of Political Philosophy 24.2 (2016): 162-182.
2: Miller, James C. "A program for direct and proxy voting in the legislative process." Public choice 7.1 (1969): 107-113.