Cardinal Francesco Armellini started the building in 1517. The work was completed in 1520. For more than 40 years the building was owned by the family Armellini and then sold in 1565 to the Cesi family.
The Cesi, of Umbrian origin, became in a few years one of the emerging families of Rome, thanks to Peter, a senator of Rome and the three cardinals Paul, Peter and Pier Donato. Pier Donato Cesi was Bishop of Narni from 1546 to 1566. He moved to Rome to assist the Pope in various diplomatic functions, and after he was made a cardinal in 1570, he thoroughly restored his palace, entrusting the task to a Lombardian architect, Martino Longhi il Vecchio who incorporated, saving time and money, the existing building created for Armellini perhaps by Antonio da Sangallo the Younger.
The result of his work is visible even now.Once the successful run of the Cesi finished the palace changed owners several times. In the 19th century, two Graziolini brothers: Giovanni Battista and Giuseppe, owned the palace. The son of Giuseppe, Giulio, sold the palace in 1862 to Count Gustavo Candelori Moroni, who in 1875 bequeathed it to his son, Gustavo, and his widow, Amallia Rosati Kinsky.