High Risk Nonprofit Security Enhancement Act of 2004

High Risk Nonprofit Security Enhancement Act of 2004 - Amends the Homeland Security Act of 2002 to authorize the Secretary of Homeland Security to: (1) enter into contracts for security enhancements and technical assistance for nonprofit organizations; and (2) issue Federal loan guarantees in connection with loans made by financial institutions to nonprofit organizations for such assistance.

Directs the Secretary to designate high-risk nonprofit organizations eligible for contracts or loans based on the vulnerability of the specific site of the nonprofit to international terrorist attacks. Sets forth criteria for vulnerability determinations.

Authorizes the use of federally guaranteed funds borrowed from lending institutions for technical assistance and security enhancements.

Requires each State to designate a State agency to evaluate applications for assistance and transmit qualifying applications to the Secretary.

Directs the Secretary to: (1) select applications for assistance, giving preference to nonprofit organizations determined to be at greatest risk of international terrorist attack; and (2) execute contracts for the highest priority applicants until available funds are expended, followed by loan guarantees.

Requires nonprofit organizations granted assistance to use certified contractors selected from a list maintained by the Secretary.

Authorizes the Secretary to provide grants to local governments to offset incremental costs associated with law enforcement in areas with a high concentration of nonprofit organizations.

Establishes within the Department of Homeland Security the Office of Community Relations and Civic Affairs to administer grant programs for nonprofit organizations and local law enforcement assistance.