| [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] |
Don, locks granted due to the owner of the blocking lock releasing them
due to node failure MUST be granted with an invalidated lock value block
if the blocking mode was PW or stronger. If this is not so, then
unless dead man locks had a mechanism guaranteeing they are granted 1st
(they don't), any OpenDLM user could wind up with a consistency problem
as resources could be granted before node failure detected.
OpenDLM is less full featured than the Tru64 implementation in that
there is no support for multiple name spaces or multiple recovery
domains. For dedicated usage (e.g. OGFS is the sole user of OpenDLM in
a cluster), this is probably acceptable, as implicitly everything is in
one name space, and there exists only one recovery domain. In our own
usage we are simulating multiple name spaces by prefixing a unique per
application instance portion to the lock key as known to the
application. Having no explicit recovery domain is more problematic,
as it entangles multiple instances of our application, some of which in
principle would not even be running on the failed node.
For that reason, I'd like to see recovery domains, although specifying
this on a per lock request basis may be overkill. Associating a
recovery domain with a group lock container or a process would be
sufficient for our needs, as we have no need for separate recovery
domains within one application instance.
An interface for revalidating all the resources within a recovery domain
would be very useful, as cleaning them up retail is a PITA.
Jack Devine
-----Original Message-----
From: Zickus II, Don
Sent: Friday, April 30, 2004 10:34 AM
To: Devine, John T
Subject: FW: [ogfs-dev]RE: [Opendlm-devel] ODLM/OGFS Recovery
Jack,
You seem to be having fun with this, try the one below. :)
-Don
-----Original Message-----
From: opendlm-devel-admin@xxxxxxxxxxxxxxxxxxxxx
[mailto:opendlm-devel-admin@xxxxxxxxxxxxxxxxxxxxx] On Behalf Of Stanley
Wang
Sent: Friday, April 30, 2004 3:22 AM
To: opendlm-devel@xxxxxxxxxxxxxxxxxxxxx
Cc: opengfs-devel@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: [ogfs-dev]RE: [Opendlm-devel] ODLM/OGFS Recovery
Daniel McNeil wrote:
[snip]
>This is the same question I had many years ago when I first heard about
>persistent locking. As I have said before, a particular implementation
>of persistent locking was used to implement a cluster file system.
>
>>From what I remember (It has been more than 4 years since I worked
>on this). As it was explained to me, the whole point of persistent
>locks was that the application did not want to keep a bunch of locks
>open to be able to find out that some event has occurred which would
>require an application recovery before using the lock. Previously, the
>application would keep the locks open and if it got a invalid value
>block back, it would know that a recovery is required. With
>"persistent" locks, the application did not have to keep the lock open,
>but would still get the invalid value block if process holding the lock
>died or a node died.
>
>Re-reading the trucluster man pages I pointed to before (like this
>one):
>http://h30097.www3.hp.com/docs/cluster_doc/cluster_16/MAN/MAN3/0021____
>.HTM
>
>This implementation seems much more complicated than what I remember.
>This looks like this DLM does keep persistent locks around if they are
>invalid after the process closes them (and they get marked invalid if a
>node dies). Thus, a new lock request will see the invalid lock even
>when all previous users have gone away. I'm guessing that is what is
>meant by persistent.
>
>The more simple implementation approach I remember (as best as I can --
>at least for the node death case anyway -- and it really isn't that
>simple). If a node dies, then all new persistent locks would return
>invalid until the dlm_rd_validate() is made. Also, existing dlm locks
>could get marked invalid during dlm recovery.
>
>This implementation was then used to implement CFS. Recovery went
>something like:
> running nomally
> node dies
> DLM recovery runs - non-persistent locks recover
> persistent locks return invalid
> deadman locks from dead node(s) for each file system are granted
> start file system recovery:
> surviving nodes pick one to run log replay
> id = dlm_attach()
> replay file system log
> if validate(id) == MORE_RECOVERY
> re-start file system recovery
> since there's been another death
> dlm_validate(id) - persistent locks valid now
> other surviving nodes wait for replay
> file system recovery complete
>
> if another node dies in the middle of file system recovery
> abort current recovery and start over
>
>
>This is simpler because persistent locks returned invalid based on
>whether a node died until the validate call was made. Every node in
>the cluster did not have to replicated each lock information.
>
>The use of persistent locks prevented surviving nodes from accessing
>metadata BEFORE file system log replay had finished. We used separate
>lock domains for each file system, so file system recovery happened in
>parallel.
>
>
>Sorry for rambling, but I think I answered the question in there
>somewhere.
>
>
Thanks a lot for your help!
Persistent locks in trucluster can survive node cases if it is attached
to a "recovery domain". Although the persistent locks are open to all
the other nodes after one node death, "DLM_xxxxVALNOTVALID" could be
used as indicator of node failure. And as I understood, there are
some differences in OpenDLM:
1. No "recovery domain" support
2. After node failure event, if the new granted lock request (on a
persistent lock resource) is PW or EX, "DLM_VALNOTVALID" will NOT be
returned.
(Is that true? I got my conclusion from "valueblock()", if I miss
sth, please correct me.)
3. There is not a dedicated API to re-validate lock value block, if the
requested mode is PW or EX and LKM_VALBLK is specified, the LVB becomes
valid again.
And it seems the second issues in previous list is crucial for ODLM/OGFS
Recovery. Change the codes is easy, but I'm afraid it will break other's
work. Any comments? Especially Don :)
Best Reagrds,
Stan
--
Opinions expressed are those of the author and do not represent Intel
Corporation "gpg --recv-keys --keyserver wwwkeys.pgp.net E1390A7F"
{E1390A7F:3AD1 1B0C 2019 E183 0CFF 55E8 369A 8B75 E139 0A7F}
-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g.
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
_______________________________________________
Opendlm-devel mailing list
Opendlm-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/opendlm-devel
-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g.
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id149&alloc_id?66&opÌk
_______________________________________________
Opengfs-devel mailing list
Opengfs-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/opengfs-devel