I am sorry the idea to swap the panel was incorrect. Sounded like an obvious thing to try in case a replacement unit could be located at low cost, especially because the FSM lacks schematics or troubleshooting information for these VFD indicators. I've spent some time trying to make sense of the wiring diagrams. We do know that the two wires for the illumination circuit are connected at the back of the radio (I85) and pass through to the connector for the control panel. If the red LED back-lighting still functions properly and replacing the panel fails to eliminate the problem, one is forced to conclude that the VFD brightness is controlled by some different means. What can it be? Maybe it's through the CAN data bus and/or the digital link between the control panel and the HVAC ECU. In that case, there also must be a special power supply circuit controlled by this input, but if so, it's safe to believe that it functions correctly (in both original and replacement units). The diagnostic table in the LAN communication section does say that "Illumination light is controlled with changing of data". If true, this cannot apply to all of the adjustable illumination, because everything visible on the diagrams is controlled in the old-fashioned Subaru way, through modulating the voltage on the negative side (even if the BIU now does the modulation).
Anyway, this still allows for a possibility that either the BIU or the HVAC ECU behind the glove box could be the culprits. I am in no position, however, to suggest that you should try replacing these.