After attending Citizenship ceremony, technically you will be Australian Citizen, So, on leaving Australia on other passport, I can think of one main point to keep in mind, Does your current country supports dual citizenship? If yes then you are good, if No then you need to make sure if you can still use your passport from home country in that case, The regulations of home country might stop you from doing that.