That does look like caterpillar frass (poo). I can't completely rule out a different insect. Sorry, can't tell from a photo what variety of moth or caterpillar larvae it came from.
Added later: sorry, missed your comment about finding a caterpillar.
Products for the vegerable or flower garden that contain Bacillus thuringiensis as the main ingredient will safely deal with caterpillars. Usually a dust, but there are also liquid products.