I have a few around in the garden and at least 1 usually moves into the greenhouse over the summer months.
The only damage they cause as such is sometimes they make themselves a little hole to sit in on top of a pot so they can be down in the damp compost. That can squish a few leaves depending on how well fed and rotund your toad happens to be
If you have a pot saucer with some water in tucked away in a corner, they use that instead.