I was accussed of stealing, what do I do?

I was standing outside a plant nursery admiring the beautiful arrangements cause I was getting some ideas for my garden and the supposed owner comes by in his truck and accuses me of stealing his plants!! I was on a public street entrance, in broad daylight and had nothing but my cell phone on me. He follows me to my car, takes a photo of my license plate with his cell phone and says he's had problem with people stealing stuff and that he's calling the cops and reporting me. I left cause I was scared and now I'm worried. There were no pictures of me doing anything, no cameras and no witnesses. Do I have anything to worry about other then never giving him any of my business?