We should certainly take every verse of the bible seriously. However, we should also ensure that we do not "expound one place of Scripture, that it be repugnant to another." (article XX, 39 articles)
So, when people claim to have "the clincher" verse on either side of the debate, we should certainly listen, but we should certainly also weigh with other Scriptures.
A clincher verse for some creationists is Romans 8:20-21.
"For the creation was subjected to frustration, not by its own choice, but by the will of the one who subjected it, in hope that the creation itself will be liberated from its bondage to decay and brought into the glorious freedom of the children of God."
The argument is that this shows that all decay and death (whether animal, plant??, radioactive??) is all a result of the fall. Even the second law of thermodynamics, it's argued, is a result of the fall. Perhaps this is why it is difficult to read the "science". Before the fall there would have been an entirely different science in place. (As far as I see, perhaps even a different mathematics, as the second law of thermodynamics is essentially the result of statistical mechanics)
The clincher for the non-literalist might be Genesis 1:6-8
6 And God said, "Let there be an expanse between the waters to separate water from water." 7 So God made the expanse and separated the water under the expanse from the water above it. And it was so. 8 God called the expanse "sky." And there was evening, and there was morning—the second day.
I would argue that there are not merely temporal problems in understanding the Genesis to be a literal scientifically accurate account. There are also spatial problems. Where is this place beyond the sky where these waters may have been stored (albeit possibly temporarily). It cannot be the upper atmosphere on a literalist reading, because the known universe is within the expanse/firmament, not beyond it (the sun, moon and stars are set within the expanse, not above it in 1:14-17).
I cannot see any way in which this makes sense within a scientifically accurate topography of the cosmos. It makes perfect sense if Moses borrowed the widely accepted topography of the universe in the Ancient Near East without challenging it, because that wasn't his point.
But I want to be cautious in suggesting that it is an absolute clincher. If other factors were to convince me of the necessity of the literalist view I would, by faith, have to trust that there was some way in which this did make sense within the early topography of the universe - a universe that was certainly a very different place to the one we know now, if there was not even any second law of thermodynamics.