I learned calculus at age 5 but didn't know it until age 28 Does this need to happen to everyone?

Who would of thought (now that I am 28 years old) that the only barrier to me learning calculus is the words "integrals" and "differentiation" which had I known the meaning I actually learned when I was 5 years old. I could have built upon the awareness when I was five if I had not given into the emotional "fear" of the words themselves which divide the "expert" from the "commoner". How ridiculous it is that we wait until college for us to study calculus. And also how rediculous it is that we only teach calculus in AP classes to mostly white persons. I know that if taught to their cultural pedegogy all minorities would thrive on understanding and making sense of the how natural phenomena they have observed since the beginning of their existence are made up of bringing together things and breaking them into chunks (integration and differentiation respectively).