Re: [PATCH 0/8] opensm: Improved mkey support

[Jim Foraker <foraker1@xxxxxxxx>]

v2 is about to be posted.  It is now a 9-patch set.  Changes from v1:

. Subnet initialization behavior changed to log errors on SubnGet
timeouts, but not flag the init as failed, so that we don't heavy sweep
more than necessary
. sldd.sh modified to sync multiple files, and neighbors/guid2mkey
caches added to its default list
. Rebased against current HEAD
. Several whitespace/code format/git log cleanup fixes

     There will be a man page patch coming shortly, but I want to get
these out the door for review now.

     Jim

On Mon, 2012-06-25 at 17:54 -0700, Jim Foraker wrote:
> I'm about to post a set of patches intended to improve mkey support
> in OpenSM.  These patches have been fairly rigorously tested on a small
> fabric, and I believe are sufficiently stable for inclusion.  The
> primary intent here is threefold:
> 
> 1) Fix a multitude of edge case issues with the existing
> single-mkey-per-subnet support in OpenSM.  For instance, the current
> implementation provides no way to change an established non-zero mkey
> without rebooting or manually re-keying each CA on the entire subnet.
> 
> 2) Enable mkey protection across the fabric.  This involves not only
> setting a non-zero protection level, but also providing the SM with a
> sufficient information cache to initialize the subnet on restart without
> having to wait for mkey lease timeouts (provided one is set).
> 
> 3) Provide a basis on which to build multiple-mkey systems for OpenSM
> (be they per-host, KDF, or random) in the future.
> 
>      The patches add two new cache files: a port guid-to-mkey cache, and
> a neighboring link (port guid to port guid) cache.
>      The guid2mkey cache is used to provide a hint at the initial mkey
> for a CA during initialization.  It is a hint only; the SM is capable of
> dealing with cases where the guid2mkey cache is incorrect, although it
> may require waiting for (potentially multiple) mkey lease timeouts at
> non-zero mkey protection levels.  The guid2mkey cache is presented first
> in the patch set, as it ends up ameliorating several corner cases in a
> cleaner way than attacking them directly did.
>      The neighbors cache file provides an initial hint to the SM of what
> port guid we may expect at the opposite end of a link that is being
> initialized.  This is necessary at mkey protection level 2, where we
> cannot do the SubnGet necessary to determine the port guid to use in
> looking up an mkey hint.
>      The changes to the osm_req functions to support mkeys in patch 2
> now require plock to be held when called.  This was generally already
> the case, but there were a few spots where it was not.  In most of these
> cases, the plock is still not technically necessary, as they occur
> during hops 0/1 when DR path traversal is trivial.  I wrapped all of
> these occurrences in locks in a separate patch (#3), in order to make
> the changes more obvious and invite comment.
> 
>      Jim

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html