In this article, we develop a passive strategy for improving index investing, which we call the Correlation Diversified Portfolio Strategy. The proposed method adjusts the weight vector of the original index according to the permutation of the assets belonging to the original index. We seek the permutation of these assets in such a way that assets that have a strong correlation with many other assets are placed at the center of the permutation. By reducing the weighting of these core assets, we can build portfolios that are more diversified and have better risk-return characteristics than the original index. We solve this asset swapping problem by taking a quantum-inspired approach. Concretely, we convert this permutation problem into an unconstrained quadratic binary optimization problem and use simulated annealing on a personal computer or annealing machine to find a near optimal solution in a reasonable time. To examine the utility and computational feasibility of the proposed method, we apply it to three major indices from the United States and Japan, and we provide numerical experiments that show that portfolios constructed by the proposed method can achieve a return. higher with lower volatility compared to the original index, while their behaviors are still similar to those of the original indexes.