College Access and Affordability Act

College Access and Affordability Act - Amends the Internal Revenue Code to: (1) rename "qualified tuition and related expenses" as "qualified higher education expenses" for purposes of the Hope Scholarship and Lifetime Learning tax credits and to include as qualified expenses tuition, fees, books, supplies, and equipment required by the taxpayer, the taxpayer's spouse, or dependents for enrollment or attendance at an eligible institution of higher learning; (2) increase to $4,000 (with annual inflation adjustments after 2006) the dollar limitation on the Hope Scholarship tax credit; (3) increase to $58,000 the modified adjusted gross income threshold for reducing the allowable amount of Hope Scholarship and Lifetime Learning tax credits; and (4) increase from two to four the number of years a Hope Scholarship tax credit may be claimed.

Amends the Higher Education Act of 1965 to: (1) grant forgiveness of student loan indebtedness up to $17,500 for individuals working in certain public service positions, including math and science teachers, emergency workers, and nurses in low-income communities; (2) require the Secretary of Education (formerly, discretionary on a case-by-case basis) to award Pell Grants to certain full-time students enrolled in college degree programs; (3) establish a special transition assistance plan for certain student loan borrowers; (4) establish a grant program to encourage transfers of academic credits between institutions of higher learning; and (5) expand the discretionary authority of student financial aid administrators in regulating student loan terms.