Named for its Belgian "château de Chimay", his family was noted for its patronage of music and the arts.
One of his sisters was Élisabeth de Riquet de Caraman-Chimay, the Countess Greffulhe.On 19 May 1890 he married Clara Ward, a sixteen-year-old American heiress.
The couple had two children:
Marie Anatole Catherine Elisabeth, Comtesse de Caraman-Chimay (30 May 1891 – 13 February 1939), married Georges De Cocq in 1918
Joseph Marie Pierre Anatole Alphonse de Riquet (6 August 1894 – 18 January 1920)They were divorced on 19 January 1897, after Clara's widely publicized elopement with a gypsy violinist.