This work studies the K-user broadcast channel where each user is assisted by one of Λ caches with a cumulative memory constraint that is equal to t times the size of the library, and where each cache serves an arbitrary number of users. In this setting, under the assumption of uncoded cache placement, no prior scheme is known to achieve a sum degrees of freedom (DoF) of t + 1, other than in the uniform case where all caches serve an equal number of users. We here show for the first time that allowing an optimized memory allocation across the caches as a function of the number of users served per cache, provides for the aforementioned DoF. A subsequent index-coding based converse proves that this performance can be close to optimal for bounded values of t.