I had to think a little more before replying. I'm still sort of groping in the dark with this angle I'm taking, but I was hoping to get around how personal experiences might tie into it. I have no doubt that it's easier for children and optimists somehow, but I wonder if it's something stranger....
It's a weird tangent to how I first started considering this, but pretty much, I don't think magical feeling corresponds to power over the environment (or else conveniences should consistently lead to joy rather than boredom).
Instead, if I had to sum it up in one word, what if magic comes from interacting with others and the world in the form of "summoning"? So power that comes from dissecting and analyzing phenomena lacks that magical quality. Somehow, being able to just ask for and receive limited powers has this element of spiritual delight. That could match up with the bit about memory, wonder, and destiny all feeling magical; all of them are ways of interacting with time as if the world has given us a gift.
Obviously, that's still a rough sketch with a lot of unanswered details. It still hasn't led to results either so I can't say I'm sure it's correct, but my gut says it's promising.