Of course all Atlus games have kind of a similar gamplay, but SMT gams are more challange than most of the others. But with Soulhackers 2, you are about as free with your team as in SMT and is has some challanging enemies you need to prepare youself. But it's about the same as SMT, so let's try to find something new.

Maybe the trails games would be interresting for you. Trails of Cold Steel 1-4 or Trailf of the Sky would be two points to start into the series. Sure you have a party with different kind of strengs like in typical jprg's, but i think the battle system helps a lot if you want to try different kind of strategies on hard bosses. The Trails of Cold steel games are not just turn based games, they have a fighting area you are able to move youself to get near the enemy or get away from the and attack with range attacks. Attack could hit in different areas as well, so you strategy is not just about the skill and attacks you use, but as well about the positioning on the field. Often that helps a lot to overcome enemies. The armor and weapons you equip are also help you do skill your characters very differently. Give a fast character a lot equipment with avoid stat, and this char will probably never been hit again from physical attacks.

A lot jrpg's have it's main party and you limited to their abilites, not like in SMT, so probably a monster catching game (SMT is more or less one as well) would be good. I think the Digimon Cyber Sleuth games would proabably a good recogmondation. Yes it's a Monster Hunting/Farming game you play 3 Digimon at the same time, able ti switch them and all of them heve their elements and atrribute, which gives them advantages and disadvantages over others. I would recommend to play it in hard to get about the difficulty as a SMT game and it could be a fun experience for you.

I'm the mod of a jrpg community, maybe the people there would post more interresting titles, because of haven't play everything.