In differentially private distributed optimization, some studies mainly utilize diminishing stepsize or interaction weakening to gradually reduce sensitivity to improve the trade-off between privacy and optimality. However, these approaches often come at the expense of the convergence rate. Commonly used acceleration methods, such as Nesterov gradient method and heavy-ball method, contribute to the fast convergence, but their dependence on historical information poses challenge to the privacy analysis in accelerated differentially private algorithms. To address this, an differentially private distributed optimization algorithm is proposed, named as theDifferentially-Private algorithm based on theGradientTracking andDoubleAccelerated (DP-GTDA) method, wheredoublemeans that both distributed heavy-ball and distributed Nesterov gradient methods are introduced to accelerate convergence. Leveraging spectral radius analysis of a 4-dimensional matrix and tail probability estimation, the accelerated linear convergence to optimal solution are established in mean and almost surely. Due to the careful design in information aggregation step, the privacy level of DP-GTDA can be determined based on differential privacy over a finite time horizon. Simulations are conducted on a distributed sensing problem to verify theoretical findings.