Rabbi Dovid Feinstein notes that the “people who were found in Shushan” were not only residents. Anybody could have been found in Shushan. More to the point, anybody who did not want to be at the party merely needed to leave the city for those seven days. The Jews’ desire to participate in the feast is the primarily reason attributed by the rabbis of the Talmud (Megillah 12a) for their potential annihilation in the Purim story. Sfas Emes brings from Yalkut Esther (1048) that the leaders of the generation did escape Shushan in time to ignore the invitation to the feast, whereas some unfortunates were “found” in the city and dragged to the seat of their temptation.