You need more, not fewer males to spread out the aggression so not one fish is targeted. IME all tall body tetra are aggressive, so you need 6 or more to manage the aggression. Males will fight one another one time, and shoal another time. If you have large tankmates, they will be threatened to shoal more often. The thin body tetra, such as rummy nose, black, glo light, neon and cardinal tetra are peaceful tetra that rarely engage in male fighting. Rasboras, which look like tetra, are tight and peaceful schooler.