Your wife is not allowed to treat her brother-in-law as a mahram; he is not her mahram and, as such, he is not different from any other stranger for her as far as the rules of hijab and isolation are concerned. A mahram is someone whom one is not allowed to marry at all in any circumstance. Your wife is allowed to marry him, if, God forbid, you divorce her and her own sister is divorced, or dies. So the bar for marriage that exists presently does not mean they are mahrams to each other. It is only a temporary bar.

Furthermore, the Prophet, peace be upon him, has warned us against isolation between genders in similar relations: He said, “In-laws are akin to death!” In other words, they face a greater risk of falling to sins than strangers, because of their greater chances of meeting and interaction, and the result would be far more disastrous than death. So let us guard against all doors and leads to temptations, for as the Prophet, peace be upon him, said, “Devil runs through the blood veins of children of Adam!” And they can only protect themselves against his snares and temptations through remembrance of Allah and by complying with His orders.

In conclusion: Your wife must observe full hijab in front of her brother-in-law, and she is not allowed to isolate with him in any circumstance.