So, we started on the edge of the park (here) and walked straight to the aquaduct (here and here)? No, not exactly. At the entrance of the park there's a big greenhouse (the Grosses Gewächshaus).
Close to the aquaduct, you'll find the Mercurius (Merkur) temple. Between those two there's lots of nature and in that nature there's lots of manmade "ornaments".