Bulletfish

Reconsider what role the bullet fish is supposed to fill. It deals regular, minor damage which essentially forces you to leave the area, because you can’t do anything about it. It has full collision versus the Tadpole, so when one is attacking you, you keep coming to a complete stop and colliding.
See if you can find a more interesting role for this creature and ask yourself what the player is supposed to do when they encounter it

A problem easily solved if we could just hit them with something that either stuns or outright kills them.

There are biomods that can do something about this, namely electric discharge and (I would assume) the camouflage one.

Maybe they could be placed in areas where they would “defend” POIs from the player, to add some more difficulty and reduce the likelihood of them feeling like a random annoyance with no purpose?

My suggestion is for them to be uninterested if the durability of your vehicle is too high, like it hurts them to hit it if it’s not already damaged
This makes situations with low durability even scarier if you’re running from something