SharePoint : Limited Access

Microsoft:

The Limited Access permission level is designed to be combined with fine-grained permissions to give users access to a specific list, document library, item, or document, without giving them access to the entire site. However, to access a list or library, for example, a user must have permission to open the parent Web site and read shared data such as the theme and navigation bars of the Web site. The Limited Access permission level cannot be customized or deleted.

 NOTE   You cannot assign this permission level to users or SharePoint groups. Instead, Windows SharePoint Services 3.0 automatically assigns this permission level to users and SharePoint groups when you grant them access to an object on your site that requires that they have access to a higher level object on which they do not have permissions. For example, if you grant users access to an item in a list and they do not have access to the list itself, Windows SharePoint Services 3.0 automatically grants them Limited Access on the list, and also the site, if needed.

Click here to see more --> on this.

So don't delete Limited Access permissions from your site as it wont harm you or your site.