Exercise is an effective treatment in chronic low back pain (CLBP), but there are few studies on CLBP in the elderly, and the intervention effect is controversial. We aimed to compare the efficacy of different exercises therapy on CLBP, dysfunction, quality of life, and mobility in the elderly./r/nWe searched Web of Science, MEDLINE, Cochrane Library, Chinese National Knowledge Infrastructure, EMBASE, and PubMed from the database inception till December 31, 2022. The publication languages were Chinese and English. Randomized controlled trials (RCTs) of exercise intervention in the elderly (≥ 60 years) with CLBP were included. Two reviewers independently extracted the data and evaluated them using the Revised Cochrane Risk of Bias Tool for Randomized Trials 2 (RoB2). The pooled effect sizes on different aspects of outcome measures were calculated./r/nSixteen articles (18 RCTs) were included, comprising a total of 989 participants. The quality of included studies was relatively high. Meta-analysis results indicated that exercise therapy could improve visual analog scale (VAS) (WMD = - 1.75, 95% CI - 2.59, - 0.92, p < 0.05), Oswestry disability index (ODI) (WMD = - 9.42, 95% CI - 15.04, - 3.79, p < 0,005), short-form 36-item health survey physical composite summary (SF-36PCS) (WMD = 7.07, 95% CI 1.01, 13.14, p < 0.05), short-form 36-item health survey mental composite summary (SF-36MCS) (WMD = 7.88, 95% CI 0.09, 15.67, p < 0.05), and timed up and go test (TUG) (WMD = - 0.92, 95% CI - 2.22, 0.38, p < 0.005)./r/nExercise therapy effectively improved VAS, ODI, and SF-36 indexes in the elderly. Based on the subgroup, when designing the exercise therapy regimen, aerobics, strength, and mind-body exercise (≥ 12 weeks, ≥ 3 times/week, ≥ 60 min) should be considered carefully, to ensure the safety and effectiveness for the rehabilitation of CLBP patients. More high-quality trials are needed in future to confirm the effect of exercise on SF-36 and TUG indexes.