Sometimes in the picture for an efficient enough upscale contrasting detail is important other times the dynamic of color representation is more important. Sometimes you can get away with less color dynamic sometimes you can get away with less detail and you can scale this issue accordingly in terms of color dynamic and contrasted resolution along with other things you can reduce that can be later remastered in from the reduced data to an upscaled version.
This would also probably mean that videos would have to be standardised which could take some time if you want to do a good job before they are ready to be encoded in the more upscalable format unless they were recorded in the format.
The quest should be for the best upscalable codec not the less needed data to convince a human because humans like to push the envelope so they are not looking for the minimum picture and the conciousness might not at first notice certain subtleties of reduction but the issue builds up in the unconcious and the issue of bad picture can over time depress many people.