This building which originated from the Kota Setar Palace is a
wooden building that was built by HRH Sultan Muhammad Jiwa Zainal Adilin Muazzam Shah, the 19th Sultan of Kedah (1710-1778), who was the founder of Alor Star town in 1735. This building had undergone demolitions several times due to attacks by the Bugis (1770) and Siamese (1821).

The concrete building was completed during the reign of HRH Sultan Ahmad Tajuddin Mukarram Shah (1854-1879). His Royal Highness had once stayed in this palace with his queen, Wan Hajar Wan Ismail who was also known as "Mak Wan Besar". Therefore, this palace is also known as the "Mak Wan Besar Palace".

YTM Tunku Abdul Rahman Putra was also born adjacent to this palace (the Pagoda building, which no longer exist today) on 8 February, 1903 and once grew up in this palace while his father the late HRH Sultan Abdul Hamid and his mother Mak Che Manjalara resided there.

This palace is also renowned as the "Pelamin Palace" when the space of the palace was extended to as a pavilion and additional rooms when HRH Sultan Abdul Hamid Halim Shah wanted to marry off his five princes and princesses. This grand wedding ceremony lasted for three months in 1904. After 1941, this palace was used as a school and an office for several organizations including the office for the St. John and the Scout movements. On 25 July 1983, this palace was declared the "Kedah Royal Museum".