Hello Sunny,
You need try to use Multiple Origin Composition concept here.
You can have different System Alias pointing to different back-end system in GW System with separate RFC Destinations.
Assign these System Alias to the GW Service ( one System Alias would be defaulted ) and now using System Alias ( SAP_Origin Parameter ) Name in the URL you can specify which back-end it should hit and get data.
Getting Data from 2 ERP systems using Multi Origin
Multiple Origin Composition - SAP Gateway Foundation (SAP_GWFND) - SAP Library
Regards,
Ashwin