I'm surprised no one mention this...
We eat a lot of Ocean fish now as I live by the ocean and my partner is a fisherman. But in the old days, we ate a ton of fresh water fish. To me the best was pan fried fresh water fish like perch, or brook trout.
Now, there is so much pollution in the fresh water sources we don't eat fresh water fish much. It may be this that is causing the fish to taste funny, or farm raised fish that were released in the ponds as stock. The fishes diet can definitely impact it's taste.
My only suggestion is If you plan on eating a lot of any type of fish fresh, or ocean caught be leary of mercury poisoning. In Maine, they have a list of the fish with the highest levels of contamination and recommendations on how much to eat. Adults may not care as much, but it's important to be cautious with kids.