If we had better memories we could simply remember when concepts first occurred to us (possibly in the womb), and hence decide which concepts are innate. We could, for example, remember whether we had the concept of red before we saw any red things–or shape or number. The whole innateness controversy depends on our poor memory for early experience. (I talk about his in my book Inborn Knowledge.)

https://ndpr.nd.edu/news/inborn-knowledge-the-mystery-within/