in SQL server is a special database role that is given by default to all logins. You cannot remove a login's membership to this role because this behavior is built into SQL Server.
Let's try again to rephrase this with even more plain words: any permissions the Public database role has, are automatically granted to all database users. That's why you should never add permissions to this role.
By the way, if you want to check if you have accidentally added more permissions to this role, you can try the below query:
USE [Database Name];
GO
SELECT
DB_NAME() as DBName,
pm.[permission_name] as PermissionName,
ob.[name] as ObjectName,
pm.class_desc as ObjectType,
ob.type_desc as TypeDescription
FROM sys.database_permissions pm
INNER JOIN sys.database_principals pr ON pm.grantee_principal_id= pr.principal_id
LEFT JOIN sys.objects ob ON pm.[major_id] = ob.[object_id]
WHERE [state]='G' and major_id>=0 and pr.[name]='public' and ob.[name] IS NOT NULL;
GO
If the above query return any results, you should revise the permissions granted to the Public database role for the specific database.
Useful Information
I have recently released a brand new software tool called "DBA Security Advisor" which checks SQL Server instances for vulnerabilities using a large number of security checks. Among other, it checks if the Public database role has been granted additional permissions besides the defaults. Check it out here!