SOGoEnablePublicAccess = TRUE; means everyone has read access.
That is independant from privileges set on the calendar level.
Therefore this works as intended.

Ok, i am a little bit confused. Unless I grant Modifier Access to my Calendar i can go back and forth granting DAndT Access and then No Access again. Only when I grant Modifier Access I can not go back to fewer rights.

And the Documentation for SOGoEnablePublicAccess = true also reads a little bit different.

"Parameter used to allow or not your users to
share publicly (ie.,requiring not authentication)
their calendars and addressbooks.
Possible values are:
? YES–toallowthem
? NO–topreventthemfromdoingso
"

Sorry I did not get your problem.
As you were talking about "Modify" privilege, I assumed you added additional privileges for one user.

Now I see, that you actually can set "Respond To" and "Modify" privileges to "Public Access" in the current V3 SOGo version.
That is wrong!
And wasn't possible before.
For "Public Access" you only should be able to give "None", "View Date and Time" and "View All" privileges.

I assume that is the heart of your problem.
As you should not be able to give that privilege, in code it is not possible to revoke it either.

Can you remove that bogus ACL with the following command?
/usr/sbin/sogo-tool manage-acl remove ${USER} Calendar/${CALENDAR_ID} anonymous

Hint:
You have to restart memcached afterwards.

That is exactly my Problem.
Your Command cleaned up my ACL again. Thank you for that.