It's probably not as simple as that, since you probably wouldn't want to drink aquarium water either. In principle there wouldn't be anything wrong with it. There are presumably many fish that live in your lake.
Fert. runoff wouldn't hurt unless it was in massive quantities. In which case, you'd probably know by looking at (or smelling) the lake. NO3 and PO4 measures are useful in assesing fertilizer runoff, except in the fall when the leaves are losing their leaves and there's a lot of plant detritus decomposing on the forest floor. Runoff from this will also show high PO4.
Pesticide runoff may be some concern. Again, it depends on the concentrations. You may be able to find out what the farms around your lake (or rather, the rivers running to the lake) use and in what quantities. The municipal water comission should have a good idea of pesticide concenetrations in the water. But if you're drawing from a well, chances are that the tap has the same as the lake.
I think that the biggest concern is the potential introduction of parasites and disease from the local system. There's also the introduction of algal spores, although if your tank is well maintained, it shouldn't be a problem, algae's going to find a way into your tank no matter what. On the upside, you may also find that you pump in a fair bit of zooplankton, or free, live fish food. But it's a lot to risk.
Being curious as I am, I think that I would be tempted to try it out on a small tank to see what really happens.