Millicent residents say they are concerned for the welfare of dozens of people sleeping rough in town.