At the moment you can only control a roles "View Schedule" option as"

We would like the ability to have more flexibility for example adding the ability to see the schedules for their Team and or Queues that they are associated with.

This same logic and filtering would be needed on the "Time Off" permission filters as well, as the agents will need to be able to see who else has time off in their team so they can make a informed decision based on company policy.