This specification only defines how calendar users on the same server are able to schedule with each other -- unauthenticated users have no way to carry out scheduling operations. Access control privileges (as per Section 6) can control which of those users can schedule with others. Calendar users not wishing to expose their calendar information to other users can do so by denying privileges to specific users, or all users, for all scheduling operations, or perhaps only freebusy.
"Attendees" can also use the Schedule-Reply request header (Section 8.1) with the value set to "F" to prevent notification to an "Organizer" that a scheduling object resource was deleted. This allows "Attendees" to remove unwanted scheduling messages without any response to the "Organizer".
Servers MUST NOT expose any private iCalendar data, or WebDAV resource state information (URLs, WebDAV properties, etc.) for one calendar user to another via scheduling messages or error responses to scheduling operations. In particular, as per Section 8.1 of [RFC4918], authorization errors MUST take preference over other errors.