It is reported that:

ʿAlī b. Ḥusayn – Allāh have mercy on him – would sometimes ride to Makkah and return without entering it (meeting anyone there), and he used to sit with Aslam the freedman of ʿUmar (himself a scholar), so it was said to him, “You leave Quraysh, and instead sit with the slave of Banī ʿAdī!” So he replied, “A man [should] sit where he benefits.”

Al-Dhahabī, Siyar Aʿlām Al-Nubalāʾ 4:388.

It is reported that:

ʿAlī b. Ḥusayn – Allāh have mercy on him – would enter the masjid and cut his way through the people until he would sit in the circle of Zayd b. Aslam. Nāfiʿ b. Jubayr said to him, “Allāh forgive you, you are the sayyid of the people, yet you come traversing the people to sit with this slave.” So ʿAlī b. Ḥusayn replied, “Knowledge is to be sought and come to and looked for wherever it is.”

Ibid.