Doug Ford ‘not at all’ worried after key mayoral allies defeated