Hello shaw22,
There are couple of options which you can explore.
Option 1:
In this option I note the availability of Fiber link connectivity between sites and since both the sites are less than 1 mile so I assume the connection would be very good and ping latency would be less than 10 ms (max).
In this case you do not need identical servers to be on both the sites. What you do is create a single Cluster of ESXi hosts (hosts in primary and in secondary) all are in same cluster managed a single vCenter server. Enable HA on the cluster.
What this lets you do is to have a high availability environment. If one server goes down in the primary site loads will be distributed over other servers. This depends on the available servers or the number of the server failure the environment can sustain. If you go ahead with N+N model (equal number of servers in both sites) then if all the servers goes down in primary then all the workload will essentially be moved to secondary site.
Also to protect vCenter server I would suggest to have Heartbeat ready and configure Primary vCenter in Primary site and secondary vCenter in Secondary site.
But note that this is not a DR solution, at best this is a stretched cluster configuration which takes advantage of the case that if primary site goes down then everything will essentially move to secondary and this solution is possible only because of the assumption (fiber link and 1 mile distance)
Requirement:
In this case all you require is
1.VMware vSphere (ESXi), VMware vCenter Server, vCenter Heartbeat (optional but highly recommended to make the solution full proof)
2. You would need shared storage for this (SAN). All the servers in the primary and secondary needs to have access to the same SAN so that VM data would be available all the time.
NOTE: If the SAN is in primary site only. Then if the SAN goes down everything goes down. To solve this, ideally you should have SAN in the secondary site as well (albeit lower type) and if the SAN at primary goes down then secondary would be available with all data. So in this case there should be a data sync between the SAN.
Option 2:
You have the local DAS data in all the servers at the primary location. You configure vSphere Replication and setup a replication for the VM data from primary to secondary. In that case if a VM goes down in primary then it can be restored to the secondary site.
This is a pure replication and DR at VM level solution. Since there is no shared storage between servers then this can be taken care of.
You would need , VMware vSphere and VMware vCenter only
Option 3:
You use Site Recovery Manager to setup a proper DR solution. In this case you would servers and VM's in the primary site and they would be replicated and protected at the secondary site. If the primary site goes down then you do a failover at the secondary site.
You can use vSphere Replication for the underlying data replication or you can use SAN based replication (in this case SAN needs to be there in both the sites).
This is a proper DR solution.
You would need
VMware vSphere, VMware vCenter, Site Recovery Manager
Please note the Option 2 and Option 3 are DR solutions.
What you asked for is "if one server goes down in primary another server needs to be up at secondary", is not a DR solution that you are looking at. What you are asking can be achieved by using option 1 (stretched cluster).
In stretched cluster mode it is simple High Availability mode that we are taking advantage of. You can configure VM's affinity so that if a server in primary site goes down all the VM's running on that server would be restarted on a server which belongs to secondary site (thus achieving what you are asking for).
In any case it is always recommended to use SAN instead of local DAS simply because if you want to take full advantage of virtualization (HA, DRS, Storage DRS, etc.) you should have a shared storage. Basic idea behind virtualization is to remove dependency on the hardware. If you keep data in local storage if the server goes down all data goes down. If it is in SAN then it is available and any server can run the VM.
Copying data from local storage to the local storage of a remote site is never a viable solution (not impossible) and will never be recommended.
Hope this clears the idea. Please let me know if you need more clarification. Please go through the following documents in the meantime to get more idea.
http://www.vmware.com/files/pdf/products/vCenter/VMware-vCenter-Server-Heartbeat-Datasheet.pdf
http://www.vmware.com/files/pdf/products/SRM/VMware_vCenter_Site_Recovery_Manager_5.5.pdf
