I think that when consciousness incarnates into a physical body, it has to deal with an overwhelming amount of sensory data that it otherwise wouldn't have to. It also has to run all the processes of the human body. Memory of previous lives then gets overwhelmed.
See, imagine if I asked you now to try to recall your fifth birthday party. You might actually be able to do so. However, if at the same time, I blast you with ear-deafening music; blind you with strobe lights; strap you onto a roller-coaster ride; spray extremely strong-smelling chemicals into your nostrils. I bet you will have great difficulty recalling your fifth birthday party. You're too distracted.
Same thing with your past lives. When you're pure consciousness, you can remember them all. But when you're a human body, you get distracted by the constant flood of information from your five senses, plus the need to maintain and coordinate all the biological processes of your body, from digestion to sweating to hormone secretion to respiration to kidney filtration to hair growth to a few thousand other processes.
So you will hardly be able to recall any of your past lives.
As a matter of fact, you may even forget where you put your car keys an hour ago.
Note: at the point of physical death, your memory immediately starts returning. For example, your entire current life will flash before your eyes, in a few seconds. That's what you can remember, when you aren't distracted by your physical body.
Further note: in Buddhist meditation, it is taught that you should be keeping your body very still (except for inevitable bodily processes like breathing). And that's because even any slight physical movement, like moving your fingers or twitching your toes, will actually be distracting your mind.