Can we replace "grades" and "years" with just "courses"?

I often wonder who exactly decides that a 4 year old child should be able to do this at school, and a 10 year old child should be able to do these things, and so on? Does it make sense when we know for a fact, backed by strong scientific evidence and common sense, that each individual learns different things at a different pace? So, instead of herding same-aged humans into a room and teaching all of them same things, can we not create rooms/customized roadmaps of different courses and topics? A student can take 2 weeks to learn the same concept of algebra as is completed by another student in 4 months. Some topics are dependent on the mastery of another topic, while some run parallel. This way, learning can be structured, and yet self-paced and robust. Each student can be a master of many subjects, spend more time on difficult areas, and excel and be super creative in some areas.

This TED talk by the founder of Khan Academy, Salman Khan, touches on this issue with an excellent metaphorical example.

When we talk of diversity in our institutions, why do we limit our thoughts to ethnicity, gender and disabilities? Why can we not be inclusive about age also in the same way? Why is it taboo to be of a certain age and not be able to write or draw or solve a certain physics or a certain math problem? Within the 14 years of education that we normally allow in India from preschool to Grade 12, we can certainly allow a wider range of flexibility/fluidity that would ensure a deeper learning for everyone, so every layer of learning is sturdy to allow a stronger next layer of knowledge or skill to built on it. We now have the necessary organizing, logistical and digital processing tools that can realize such customized courses and fields frameworks. I think a major overhaul of the education system is long overdue, and every educationist needs to be thinking of this issue so that we can come together to create a workable solution.