On March 30, 1895, the members of Alpha Delta Phi moved into their new fraternity house on the site of the current Perry House. Since then many changes and additions have occurred that have enlarged the structure. In 1906 the lots on either side of the fraternity were bought and added to the fraternity's holdings. In that same year the lodge added the 'goat' room. In 1914 the house was remodeled and a library was added to the east side and a memorial porch was added in the memory of one of the brothers. This porch replaced the old one and is still a part of the house today. The library was donated by Howard Eells (Class of 1915) in memory of Samuel Eells (Class of 1917). A final wing was added in 1929, which brought the house to the size which it remains today and enabled the fraternity to house twenty-five people. Towards the back of the building is the newest part of the house, added in 1970. This addition has helped to house more students.
In 1964, the Alpha Delta Phi house was renamed in honor of Arthur Latham Perry and his five sons: Lewis, Bliss, Arthur, Walter and Carroll. Four of the sons had graduated Williams College and were members of the Alpha Delta Phi fraternity.
By Patrick J. McCurdy