Photographers Rights

As a photographer I find it difficult to capture images in public without being questioned by strangers or stared at. It seems people get a bit paranoid when they see someone with a camera. I shoot a lot of images for stock and am always looking for the perfect location. Occasionally, I will pull my car over to the side of the road to capture a beautiful sunset or a duck floating on a pond. I really can't understand why the public finds photographers so threatening.