Masjid Haji Muhammad Salleh sits on piece of waqaf land within the bustling Central Business District. The mosque is flanked to the left by the elevated East Coast Parkway and on its right by Mount Palmer on which sit two important shrines of Habib Noh and Habib Abdul Rahman.

The mosque was built by Haji Muhammad Salleh a merchant from Batavia, now known as Jakarta, who lived in Singapore in the mid 19th to the early 20th century. Haji Muhammad Salleh was a friend of Habib Noh, whom he highly revered. Habib Noh frequented Mount Palmer to pray in seclusion and Haji Muhammad Salleh intended to build a surau there for his beloved friend to pray in ease however Habib Noh passed away before his wish was fulfilled.

Haji Muhammad Salleh donated a piece of land at the foot of Mount Palmer and began construction of a mosque to serve the visitors of the shrine. The mosque was completed in 1903.

Two decades on, the mosque which was designed to accommodate 500 worshippers, now attract more than 1200 worshippers for the Friday and Eid prayers that tents have been erected as makeshift roof. For the Haul of Habib Noh, the Mosque and the adjoining Maqam Habib Noh now receive more than 8000 visitors.