distributed cluster

[Jerker Nyberg <jerker@xxxxxxxxxxxx>]

This may not really be a subject ceph-devel mailinglist but rather a 
potential ceph-users? I hope it is ok to write here. I would like to 
discuss the if it sounds reasonable to run a Ceph cluster distributed over 
a metro (city) network.

Let us assume we have a couple of sites distributed over a metro network 
with at least gigabit interconnect. The demands for storage capacity and 
speed at our sites are increasing together with the demands for reasonably 
stable storage. May Ceph be a port of a solution?

One idea is to set up Ceph distributed over this metro network. A public 
service network is announced at all sites, anycasted from the storage 
SMB/NFS/RGW(?)-to-Ceph gateway. (for stateless connections). Statefull 
connections (iSCSI?) has to contact the individual storage gateways and 
redundancy is handled at the application level (dual path). Ceph kernel 
clients contact the storage servers directly.

Hopefully this means that clients at the sites with a storage gateway will 
contact it. Clients at a site without a local storage gateway, or when the 
local gateway is down, will contact a storage gateway at another site.

Hopefully not all power and network at the whole city will go 
down at once!

Does this sound reasonable? It should be easy to scale up with more 
storage nodes with Ceph. Or is it better to put all servers in the same 
server room?

                        Internet
                         |   |
                        Routers
                         |   |
   Metro network  =============================
                  |     |     |     |    |    |
   Sites          R     R     R     R    R    R
                  |     |     |     |
   Servers      Ceph1 Ceph2 Ceph3 Ceph4


--jerker
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html