Yes, there is plenty of evidence that infant mortality was high. 

However this does not mean their adult life-expectancy was comparable to
ours.  After surviving childhood they remained vulnerable to the many
infectious diseases and other diseases we have since cured or prevented with
modern medicine, to say nothing of death from other causes related to their
perilous life-styles as nomadic hunters. 

Life expectancy for mature adults beyond childhood here in the US has jumped
something like 10 years in the last 100 years. I'm sure this same trend has
been in place since ancient times because of improvements not only in
medicine, but also sanitation and nutrition. 

I'm saying here that I think it is unrealistic to regard Paleolithic people
as some kind of god-like pinnacles of health and longevity. Certainly they
were healthy, but I think it's a fantasy to put them up on a pedestal. Even
something like a very seriously broken leg would have been potentially
lethal to a paleo. He would have been left behind by the tribe, much like
Native American tribes were known to leave their sick and elderly behind to
die. 

The most compelling evidence in favor of paleodiet theory comes from
comparing paleos to the decrepit and pitifully unhealthy early farmers of
the Neolithic period. The comparison to moderns is much less convincing.
Advancements in medical science, nutrition, and hygiene have largely made up
for the mistakes of the earliest farmers. However this does not mean a
paleolithic diet will not make modern people even healthier and
longer-lived. Obviously I think it should or I wouldn't be here.

-gts