The article
Bleses, D., Vach, W., Slott, M., Wehberg, S., Thomsen, P., Madsen, T. O., & Basbøll, H. (2008). Early vocabulary development in Danish and other languages: A CDI-based comparison. Journal of child language, 35(3), 619-650.
compares several studies that measured how fast children acquire languages by various measures. It argues that languages with plenty of vowel sounds are harder for children to learn. Some comparable languages in the study are Danish and Swedish, and UK and USA English. Danish was harder than Swedish, and UK English harder than USA English.
Of these, the Englishes have similar grammar, while Danish has, at least on surface, simpler grammar than Swedish.
This would suggest that grammar is not a deciding factor on how difficult a language is to learn as a child.
A more throughout answer would use an actual measure of grammatical complexity and check the included languages for that.
Anyone interested in the subject is encouraged to read the paper and see if there are more recent studies in the same vein; Searching for the article on Google scholar and seeing what refers to it would be a good place to begin.