Welcome ot the Jungle....
I have the issue from time to time with my container plants...
While I treat my containers in the fall with an insecticide drench some critters slip under the radar.
Treat your soil with an inexpencive insecticide as a soil drench...
Use a saucer to catch the excess.....repeat as needed.
And of course remove the food source .