Switching between azure/on-prem specific assemblies at runtime or pre-build based on config

I have a wcf web service which could be deployed to azure or on-prem.
I am trying to have a common code base for both scenarios but with different web.config for each sceanrio.
The problem is that when the wcf service is deployed on azure it needs to talk to azure cache instance and if deployed on prem it will talk to windows app fabric cache server.
the code to talk to both remains the same because they are the same caching product essentially.
However - if the wcf service is going to talk to azure cache then the project should have a reference to the following assemblies:
Microsoft.ApplicationServer.Caching.Core.dll (1.0.4817.0)
Microsoft.WindowsFabric.Common.dll (1.0.5627.0)
Microsoft.ApplicationServer.Caching.Client.dll (1.0.4817.0)
Microsoft.WindowsFabric.Data.Common.dll
Microsoft.ApplicationServer.Caching.AzureCommon.dll
Microsoft.ApplicationServer.Caching.AzureClientHelper.dll

if the code has to talk to app fabric on prem then the following assemblies have to be referenced.
Microsoft.ApplicationServer.Caching.Core.dll (1.0.4632.0)
Microsoft.WindowsFabric.Common.dll (1.0.4619.0)
Microsoft.ApplicationServer.Caching.Client.dll (1.0.4632.0)
Microsoft.WindowsFabric.Data.Common.dll

I am looking for a solution so that I can keep the same code base (.cs files and .csproj) and with a configuration switch in web.config it should refer to the right assemblies,
for example if 'UseAzure = 1' in web.config then the azure cache client side dlls should be referenced and loaded and if 'UseAzure = 0' then AF on-Prem dlls should be loaded.
I am wondering if this is possible using run time C# code or via a visual studio pre/post build task.
I am open to any other ideas to acheive this.
Could someone confirm on the feasability of these two options and if so point me to some code which does something similar.