PriorityQueue is a Queue and it provides the functionality of FIFO data structure, while TreeSet is a Set.
TreeSet is a Set that doesn't allow duplicate elements but PriorityQueue may contain duplicates.
The PriorityQueue provides largest or smallest element in O(1) time, which is not possible by TreeSet. Since TreeSet is backed by a red-black tree, the search operation will take O(log N) time.
TreeSet have all elements remain in the sorted order, while in priority queue apart from root, which is guaranteed to be smallest or largest depending upon Comparing logic, rest of element may or may not follow any ordering.