The villa was built after 1700 by marquis Paolo Antonio Menafoglio, who conceived it as a leisure residence, surrounded by an Italian style garden. After the financial decay of the family, in 1823, the villa was ceased to Pompeo Litta Visconti Arese, a milanese aristocrat who appointed the well known architect Luigi Canonica of the enlargement of the villa, including some modifications of the garden, according to the new English romantic taste. Since 1956, the villa has bee the seat of an important contemporary art collection, gathered by the new owner, count Giuseppe Panza di Biurno. The count's taste blends American pop art with refined furnitures of the 16th-19th centuries and African and Pre-colombian pieces.

The palace was built in a Lombard "Barocchetto" style with Neoclassical influences, as a leisure residence for Francesco III D'Este, Duke of Modena and Reggio Emilia. After the Duke's death and due to the high maintenance costs, the palace changed various owners until it was bought in 1837 by Carlo Pellegrini Robbioni, who made some modifications to the garden, according to the English romantic taste of the age. In 1882 the mansion was acquired by Varese City Hall and open to the public the following year.