[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [ogfs-dev]Initial patch for integreating OpenDLM with OpenGFS



On Thursday 04 September 2003 07:48 am, Stanley Wang wrote:
> Hi, Folks
> Sorry for later posting this patch (I've promised to post it at the end
> of Aug).
> I have to say that the debuging work is nightmare :( I use HA heartbeat
> as the cluster mgr. It's a good choice, simple and stable. Using OpenDLM
> is also a simple work, but unfortunately it is not as stable as the
> "heartbeat". It brought me much trouble :( 
> I'm now still blocked by some bugs(OpenDLM's and mine) and trying to fix
> them.
> Following is a very initial patch to support OpenDLM. I succeeded to
> touch a file with this patch when disabling the deadman lock. I'm
> chasing bugs and hope this patch would work properly soon. 
> Bug fixing codes will also be posted to the list continually. I'll check
> in codes to the cvs when the deadman lock works at least.
> There are also some pitfalls in design, but it is not hard to fix.
> If you have time, please give me feedback and suggestion :)
> 
> BTW, I'll write the design doc and howto.
> 
> Best Regards,
> Stan

I haven't tested this yet, but here are some comments jsut form looking at it 
(mostly nitpicks)

> 
> -- 
> Opinions expressed are those of the author and do not represent Intel
> Corporation
> "gpg --recv-keys --keyserver wwwkeys.pgp.net E1390A7F"
> {E1390A7F:3AD1 1B0C 2019 E183 0CFF  55E8 369A 8B75 E139 0A7F}
> 
> 
> diff -Naur -X /usr/src/dontdiff opengfs-old/src/include/lm_interface.h
> opengfs-bak/src/include/lm_interface.h
> --- opengfs-old/src/include/lm_interface.h	2003-09-04 20:04:26.000000000
> +0800
> +++ opengfs-bak/src/include/lm_interface.h	2003-09-04 16:26:12.000000000
> +0800
> @@ -66,6 +66,7 @@
>  #define LM_TYPE_PLOCK          (0x09)
>  #define LM_TYPE_PLOCK_HEAD     (0x0A)
>  #define LM_TYPE_LVB_MASK       (0x80)
> +#define LM_TYPE_DEADMAN	       (0x0B)
>  
>  /*  States passed to lock()  */
>  
> diff -Naur -X /usr/src/dontdiff
> opengfs-old/src/locking/modules/Makefile.am
> opengfs-bak/src/locking/modules/Makefile.am
> --- opengfs-old/src/locking/modules/Makefile.am	2003-09-04
> 20:05:28.000000000 +0800
> +++ opengfs-bak/src/locking/modules/Makefile.am	2003-09-04
> 16:26:16.000000000 +0800
> @@ -1,2 +1,2 @@
>  # -*-Makefile-*-
> -SUBDIRS = nolock memexp stats
> +SUBDIRS = nolock memexp stats ogfs_dlm
> diff -Naur -X /usr/src/dontdiff
> opengfs-old/src/locking/modules/ogfs_dlm/cluster_config.h
> opengfs-bak/src/locking/modules/ogfs_dlm/cluster_config.h
> --- opengfs-old/src/locking/modules/ogfs_dlm/cluster_config.h	1970-01-01
> 08:00:00.000000000 +0800
> +++ opengfs-bak/src/locking/modules/ogfs_dlm/cluster_config.h	2003-09-04

I would prefer it to go in a directory called opendlm, it would follow what's 
already there, and allow other dlm's (lustre's for example) without being too 
confusing.

> 16:26:18.000000000 +0800
> @@ -0,0 +1,57 @@
> +/*
> + *
> + *    Copyright 1999-2001 Sistina Software, Inc.
> + *
> + *    This is free software released under the GNU General Public
> License.
> + *    There is no warranty for this software.  See the file COPYING for
> + *    details.
> + *
> + *    See the file AUTHORS for a list of contributors.
> + *
> + *    This file was maintained by:
> + *      David C. Teigland <teigland@sistina.com>
> + *
> + */

If you copied this file's contents from another file then it should be 
copyright opengfs project with portions copyrighted by sistina. otherwise it 
should be copyrighted just by the opengfs project

> +#ifndef _CLUSTER_CONFIG_H
> +#define _CLUSTER_CONFIG_H
> +
> +//#include "ogfs_ondisk.h"
> +#include "global.h"
> +
> +#define OGFS_LOCKNAME_LEN	(64)
> +
> +#define OGFS_CLUSTER_GLOBAL  (0x47465341)
> +#define OGFS_CLUSTER_NODE    (0x47465342)
> +
> +#define OGFS_CLUSTER_VERSION (0x47010000)
> +
> +#define CIDEV_BLOCKSIZE     (4096)
> +
> +/*  First sector of cidev  */
> +
> +struct cluster_global {
> +	uint32	cg_magic;	/* magic number */
> +	uint32	cg_version;	/* version number */
> +	char	cg_lockdev[64];	/* lock device */
> +	char	cg_datadev[64];	/* data device */
> +	uint32	cg_cbport;	/* call-back port */
> +	uint32	cg_node_timeout;	/* in seconds */
> +};
> +typedef struct cluster_global cluster_global_t;
> +
> +/*  One struct per sector.  Implicit: JID = block - 1  */
> +
> +struct cluster_node {
> +	uint32	cn_magic;	/* Magic number */
> +	uint32	cn_ipaddr;	/* IP address of node */
> +	uint32	cn_cid;		/* Client ID of node */
> +	uint32	cn_num_stomith;	/* Number of stomith structs */
> +};
> +typedef struct cluster_node cluster_node_t;
> +
> +/*  On-disk format translation functions  */
> +
> +void cluster_global_in(struct cluster_global *cg, char *buf);
> +void cluster_node_in(struct cluster_node *cn, char *buf);
> +
> +#endif /* _CLUSTER_CONFIG_H */
> diff -Naur -X /usr/src/dontdiff
> opengfs-old/src/locking/modules/ogfs_dlm/cluster_ondisk.c
> opengfs-bak/src/locking/modules/ogfs_dlm/cluster_ondisk.c
> --- opengfs-old/src/locking/modules/ogfs_dlm/cluster_ondisk.c	1970-01-01
> 08:00:00.000000000 +0800
> +++ opengfs-bak/src/locking/modules/ogfs_dlm/cluster_ondisk.c	2003-09-04
> 16:26:18.000000000 +0800
> @@ -0,0 +1,66 @@
> +/*
> + *
> + *    Copyright 1995-2001 Regents of the University of Minnesota
> + *
> + *    This is free software released under the GNU General Public
> License.
> + *    There is no warranty for this software.  See the file COPYING for
> + *    details.
> + *
> + *    See the file AUTHORS for a list of contributors.
> + *
> + *    This file was maintained by:
> + *      David C. Teigland  <teigland@sistina.com>
> + *
> + */
> +
> +#include "config.h"
> +
> +#ifdef __KERNEL__
> +#include <linux/module.h>
> +#include <linux/string.h>
> +#include "osi.h"
> +#else /* __KERNEL__ */
> +#include <string.h>
> +#include "osi_endian.h"
> +#endif /* __KERNEL__ */
> +
> +#include "cluster_config.h"
> +
> +#define CPIN_8(x, y, z) {memcpy((x), (y), (z));}
> +#define CPOUT_8(x, y, z) {memcpy((y), (x), (z));}
> +#define CPIN_16(x, y) {(x) = be16_to_cpu((y));}
> +#define CPOUT_16(x, y) {(y) = cpu_to_be16((x));}
> +#define CPIN_32(x, y) {(x) = be32_to_cpu((y));}
> +#define CPOUT_32(x, y) {(y) = cpu_to_be32((x));}
> +#define CPIN_64(x, y) {(x) = be64_to_cpu((y));}
> +#define CPOUT_64(x, y) {(y) = cpu_to_be64((x));}
> +
> +
> +/*  Nothing STOMITH-specific below  */
> +
> +void
> +cluster_global_in(cluster_global_t * cg, char *buf)
> +{
> +	cluster_global_t *bufcg = (cluster_global_t *) buf;
> +
> +	CPIN_32(cg->cg_magic, bufcg->cg_magic);
> +	CPIN_32(cg->cg_version, bufcg->cg_version);
> +
> +	CPIN_8(cg->cg_lockdev, bufcg->cg_lockdev, OGFS_LOCKNAME_LEN);
> +	CPIN_8(cg->cg_datadev, bufcg->cg_datadev, OGFS_LOCKNAME_LEN);
> +
> +	CPIN_32(cg->cg_cbport, bufcg->cg_cbport);
> +	CPIN_32(cg->cg_node_timeout, bufcg->cg_node_timeout);
> +}
> +
> +void
> +cluster_node_in(cluster_node_t * cn, char *buf)
> +{
> +	cluster_node_t *bufcn = (cluster_node_t *) buf;
> +
> +	CPIN_32(cn->cn_magic, bufcn->cn_magic);
> +	CPIN_32(cn->cn_ipaddr, bufcn->cn_ipaddr);
> +	CPIN_32(cn->cn_cid, bufcn->cn_cid);
> +	CPIN_32(cn->cn_num_stomith, bufcn->cn_num_stomith);
> +}
> +
> diff -Naur -X /usr/src/dontdiff
> opengfs-old/src/locking/modules/ogfs_dlm/deadman.c
> opengfs-bak/src/locking/modules/ogfs_dlm/deadman.c
> --- opengfs-old/src/locking/modules/ogfs_dlm/deadman.c	1970-01-01
> 08:00:00.000000000 +0800
> +++ opengfs-bak/src/locking/modules/ogfs_dlm/deadman.c	2003-09-04
> 19:50:02.000000000 +0800
> @@ -0,0 +1,167 @@
> +/* -*- linux-c -*- */
> +
> +#include <linux/module.h>
> +#include <linux/version.h>
> +#include <linux/init.h>
> +#include <linux/kmod.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +#include <linux/list.h>
> +
> +
> +#include <lm_interface.h>
> +#include "ogfs_dlm.h"
> +
> +#define OGFS_DLM_SUCCESS	0
> +#define OGFS_DLM_FAILED		1
> +#define OGFS_DLM_NONFIRST	2
> +
> +struct deadman_lock {
> +	struct list_head	list;
> +	struct opendlm_lock	lock;
> +};
> +
> +void ignore_bast(void *bastargs, int mode)
> +{
> +	return;
> +}
> +
> +void ignore_ast(void *bastargs)
> +{
> +	return;
> +}
> +
> +void ignore_unlockast(void *unlockastargs, int lstat, void* extrap)
> +{
> +	return;
> +}
> +
> +void other_node_die(void* bastargs)
> +{
> +	struct opendlm_lock *lock = (struct opendlm_lock *)bastargs;
> +	struct ogfs_dlm *dlm = find_dlm(lock->lockname.name_space);
> +
> +	if(!dlm) {
> +		printk("%s:  Can not find failed node!\n", PROTO_NAME);
> +		return;
> +	}
> +	
> +	dlm->cb(dlm->fsdata, LM_CB_EXPIRED, 
> +		(void *)(&lock->lockname.lock_num));
> +}
> +
> +void deadman_block_others(void *bastargs, int mode)
> +{
> +	struct opendlm_lock *lock = (struct opendlm_lock *)bastargs;
> +	
> +	dlmunlock(lock->lksb.lockid, NULL, LKM_FORCE, ignore_unlockast, NULL,
> NULL);
> +	dlmlock(LKM_CRMODE, &lock->lksb, LKM_NOQUEUE, (void
> *)(&lock->lockname),
> +			 sizeof(struct opendlm_lockname), other_node_die,
> +			 (void *)lock, ignore_bast);
> +	
> +	return;
> +}
> +
> +void cleanup_deadman_list(struct ogfs_dlm *dlm)
> +{
> +	struct deadman_lock *tmp;
> +
> +	list_for_each_entry(tmp, &dlm->deadman, list){
> +		dlmunlock(tmp->lock.lksb.lockid, NULL, LKM_FORCE, 
> +				ignore_unlockast, NULL, NULL);
> +		list_del(&tmp->list);
> +		kfree(tmp);
> +	}
> +}
> +
> +
> +int init_deadman(struct ogfs_dlm *dlm, int first)
> +{
> +	unsigned int lock_num;
> +	struct deadman_lock *deadman;
> +	dlm_stats_t status;
> +
> +	for(lock_num = 0; lock_num < dlm->nodes_count; lock_num++){
> +		deadman = (struct deadman_lock *)kmalloc(sizeof(struct deadman_lock),
> GFP_KERNEL);
> +		if(!deadman)
> +			return OGFS_DLM_FAILED;
> +
> +		memset(deadman, 0, sizeof(struct deadman_lock));
> +
> +		strncpy(deadman->lock.lockname.magic, OGFS_DLM_MAGIC,
> OGFS_DLM_MAGICLEN);
> +		deadman->lock.lockname.name_space = dlm->name_space;
> +		deadman->lock.lockname.lock_num = (unsigned long long)lock_num;
> +		deadman->lock.lockname.lock_type = LM_TYPE_DEADMAN;
> +	
> +		if(lock_num != dlm->jid){
> +			if(first){
> +				status = my_dlmlock_sync(LKM_EXMODE, &deadman->lock.lksb,
> LKM_NOQUEUE, &deadman->lock,
> +							deadman_block_others);
> +			} else {
> +				status = dlmlock(LKM_CRMODE, &deadman->lock.lksb, LKM_NOQUEUE,
> (void *)(&deadman->lock.lockname),
> +							sizeof(struct opendlm_lockname), other_node_die,
> +							(void *)&deadman->lock, deadman_block_others);
> +			}
> +		} else {
> +			status = my_dlmlock_sync(LKM_EXMODE, &(deadman->lock.lksb), 0,
> &(deadman->lock), ignore_bast);
> +		}
> +
> +		if(status != DLM_NORMAL){
> +			kfree(deadman);
> +			return OGFS_DLM_FAILED; 
> +		}
> +
> +		if(first && deadman->lock.lksb.status != DLM_NORMAL){
> +			kfree(deadman);
> +			return OGFS_DLM_NONFIRST;
> +		}
> +
> +		list_add(&deadman->list, &dlm->deadman);
> +	}
> +
> +	return OGFS_DLM_SUCCESS;
> +}
> +
> +int start_deadman_lock(struct ogfs_dlm *dlm, unsigned int *first)
> +{
> +	int error;
> +	
> +	error = grab_mount_lock(dlm);
> +	if(error)
> +		return -1;
> +
> +	error = init_deadman(dlm, 1);
> +
> +	switch (error) {
> +
> +	case OGFS_DLM_FAILED:
> +		cleanup_deadman_list(dlm);
> +		release_mount_lock(dlm);
> +		return -1;
> +
> +	case OGFS_DLM_NONFIRST:
> +		cleanup_deadman_list(dlm);
> +		release_mount_lock(dlm);
> +		break;
> +	case OGFS_DLM_SUCCESS:
> +		*first = TRUE;
> +		return 0;
> +	}
> +
> +	*first = FALSE;
> +	error = init_deadman(dlm, 0);
> +
> +	if(error == OGFS_DLM_FAILED){
> +		cleanup_deadman_list(dlm);
> +		release_mount_lock(dlm);
> +		return -1;
> +	}
> +
> +	return 0;
> +}
> +
> +int stop_deadman_lock(struct ogfs_dlm *dlm)
> +{
> +	cleanup_deadman_list(dlm);
> +	return 0;
> +}
> diff -Naur -X /usr/src/dontdiff
> opengfs-old/src/locking/modules/ogfs_dlm/dlm.c
> opengfs-bak/src/locking/modules/ogfs_dlm/dlm.c
> --- opengfs-old/src/locking/modules/ogfs_dlm/dlm.c	1970-01-01
> 08:00:00.000000000 +0800
> +++ opengfs-bak/src/locking/modules/ogfs_dlm/dlm.c	2003-09-04
> 19:47:14.000000000 +0800
> @@ -0,0 +1,287 @@
> +/* -*- linux-c -*- */

this file should have a copyright/license notice

> +
> +#include <linux/module.h>
> +#include <linux/version.h>
> +#include <linux/init.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +#include <linux/kmod.h>
> +#include <linux/list.h>
> +
> +#include <lm_interface.h>
> +#include "ogfs_dlm.h"
> +
> +LIST_HEAD(dlm_list);
> +struct lm_lockops opendlm_ops;
> +
> +lm_lockspace_t *opendlm_mount(char *table_name, char *host_data,
> lm_callback_t cb, 
> +			      lm_fsdata_t * fsdata,unsigned int *jid, unsigned int *first)
> +{
> +	struct ogfs_dlm *dlm;
> +	char name[256];
> +	uint32 ip;
> +	uint32 hash;
> +	int error;
> +
> +	MOD_INC_USE_COUNT;
> +
> +	name[255] = '\0';
> +	strncpy(name, table_name, 255);
> +	hash = locktable_hash(table_name, strlen(table_name));
> +
> +	inet_aton(host_data, &ip);
> +
> +	/*  Check to make sure this lock table isn't already being used  */
> +	dlm = find_dlm(hash);
> +	if(dlm) {
> +		printk("%s:  \"%s\" encount a name space collsion\n", PROTO_NAME,
> +		       name);
> +		goto fail;
> +	}
> +
> +	/*  Set up our main structure  */
> +
> +	dlm = (struct ogfs_dlm *)kmalloc(sizeof(struct ogfs_dlm), GFP_KERNEL);
> +	if (!dlm) {
> +		printk("%s:  out of memory\n", PROTO_NAME);
> +		goto fail;
> +	}
> +	memset(dlm, 0, sizeof(struct ogfs_dlm));
> +
> +	strcpy(dlm->table_name, name);
> +
> +	dlm->cb = cb;
> +	dlm->fsdata = fsdata;
> +	dlm->name_space = hash;
> +	INIT_LIST_HEAD(&dlm->deadman);
> +	
> +	dlm->cfile = filp_open(name, O_RDONLY, 0);
> +	if (IS_ERR(dlm->cfile)) {
> +		printk("%s: open error %ld\n", PROTO_NAME, PTR_ERR(dlm->cfile));
> +		dlm->cfile = NULL;
> +	}
> +	if (!dlm->cfile) {
> +		printk("%s:  can't open cidev %s\n", PROTO_NAME, name);
> +		goto fail_free_dlm;
> +	}
> +
> +	if (!S_ISBLK(dlm->cfile->f_dentry->d_inode->i_mode)) {
> +		printk("%s: cidev %s is not a blockdevice\n", PROTO_NAME, name);
> +		goto fail_free_dlm;
> +	}
> +
> +	/*  Get information from cidev  */
> +	error = setup_jid(dlm, ip);
> +	if (error) {
> +		printk("%s:  bad parameters from %s\n", PROTO_NAME, name);
> +		goto fail_dput;
> +	}
> +	*jid = dlm->jid;
> +
> +	/*  Wait for possible mount state recovery  */
> +	error = start_deadman_lock(dlm, first);
> +	if (error < 0) {
> +		printk("%s:  start deadman lock failed (%d)\n", PROTO_NAME,
> +		       error);
> +		goto fail_dput;
> +	}
> +
> +	list_add(&dlm->list, &dlm_list);
> +
> +	return ((lm_lockspace_t *) dlm);
> +
> +fail_dput:
> +	filp_close(dlm->cfile, NULL);
> +
> +fail_free_dlm:
> +	kfree(dlm);
> +
> +fail:
> +	MOD_DEC_USE_COUNT;
> +	return (NULL);
> +}
> +
> +void opendlm_others_may_mount(lm_lockspace_t * lockspace)
> +{
> +	/*unlock mount lock*/
> +	release_mount_lock((struct ogfs_dlm *)lockspace);
> +}
> +
> +void opendlm_unmount(lm_lockspace_t * lockspace)
> +{
> +	struct ogfs_dlm *dlm = (struct ogfs_dlm *)lockspace;
> +	struct ogfs_dlm *tmp;
> +	int done = FALSE;
> +
> +	list_for_each_entry(tmp, &dlm_list, list)
> +		if (tmp == dlm) {
> +			list_del(&dlm->list);
> +			done = TRUE;
> +		}
> +
> +	if(!done) {
> +		printk("%s:  given a weird lockspace pointer\n", PROTO_NAME);
> +	} else {
> +		stop_deadman_lock(dlm);
> +		filp_close(dlm->cfile, NULL);
> +		kfree(dlm);
> +		MOD_DEC_USE_COUNT;
> +	}
> +}
> +
> +lm_lock_t *opendlm_get_lock(lm_lockspace_t * lockspace, lm_lockname_t *
> name)
> +{
> +	struct opendlm_lock *lock;
> +	struct ogfs_dlm *dlm = (struct ogfs_dlm *)lockspace;
> +
> +	lock = (struct opendlm_lock *)kmalloc(sizeof(struct opendlm_lock),
> GFP_KERNEL);
> +	if(!lock)
> +		goto lock_failed;
> +
> +	memset(lock, 0, sizeof(struct opendlm_lock));
> +
> +	lock->lvb = (lm_lvb_t *)kmalloc(sizeof(lm_lvb_t), GFP_KERNEL);
> +
> +	if(lock->lvb == NULL)
> +		goto lvb_failed;
> +
> +	lock->lvb->lvb_data = (char *)&lock->lksb.value;
> +	lock->lvb->lvb_length = MAXLOCKVAL;
> +	
> +	strncpy(lock->lockname.magic, OGFS_DLM_MAGIC, OGFS_DLM_MAGICLEN);
> +	lock->lockname.name_space = dlm->name_space;
> +	lock->lockname.lock_type = name->ln_type;
> +	lock->lockname.lock_num = name->ln_number;
> +	lock->dlm = dlm;
> +	
> +	return (lm_lock_t *)lock;
> +
> + lvb_failed:
> +	kfree(lock);
> + lock_failed:
> +	return NULL;
> +}
> +
> +void opendlm_put_lock(lm_lock_t * lk)
> +{
> +	struct opendlm_lock *lock = (struct opendlm_lock *)lk;
> +	kfree(lock->lvb);
> +	kfree(lock);
> +}
> +
> +int opendlm_lock(lm_lock_t * lk, unsigned int cur_state, unsigned int
> req_state, 
> +		 unsigned int flags)
> +{
> +	dlm_stats_t status;
> +	struct opendlm_lock *lock = (struct opendlm_lock *)lk;
> +	int node_id = lock->dlm->jid + 1; /* node_id = jid+1; specified in
> config file*/
> +	int lockmode = 0;
> +	int lockflags = 0;
> +
> +	parse_lockreq(cur_state, req_state, &lockmode, &lockflags);
> +	
> +	if (flags | LM_FLAG_TRY){
> +		lockflags |= LKM_NOQUEUE;
> +	}
> +	
> +	if (flags | LM_FLAG_NOEXP) {
> +		dlm_purge(node_id, 0, 0);
> +	}
> +
> +	lockflags |= LKM_VALBLK;
> +
> +	status = my_dlmlock_sync(lockmode, &lock->lksb, lockflags, (void
> *)lock, block_others);
> +
> +	if(status != DLM_NORMAL)
> +		goto fail; 
> +
> +	if(lock->lksb.status != DLM_NORMAL)
> +		goto fail;
> +
> +	return req_state;
> +
> + fail:
> +	return -1;
> +}
> +
> +int opendlm_unlock(lm_lock_t * lk, unsigned int cur_state)
> +{
> +	struct opendlm_lock *lock = (struct opendlm_lock *)lk;
> +
> +	dlmunlock(lock->lksb.lockid, lock->lvb->lvb_data,
> LKM_FORCE|LKM_VALBLK,
> +			ignore_unlockast, NULL, NULL);
> +
> +	return 0;
> +}
> +
> +int opendlm_reset(lm_lock_t * lk)
> +{
> +        struct opendlm_lock *lock = (struct opendlm_lock *)lk;
> +
> +        dlmunlock(lock->lksb.lockid, NULL, LKM_FORCE|LKM_VALBLK, 
> +			ignore_unlockast, NULL, NULL);
> +
> +        return 0;
> +}
> +
> +void opendlm_cancel(lm_lock_t * lk, int on)
> +{
> +        struct opendlm_lock *lock = (struct opendlm_lock *)lk;
> +	
> +	lock->cancel = on ? 1 : 0;
> +}
> +
> +lm_lvb_t *opendlm_hold_lvb(lm_lock_t * lk)
> +{
> +	struct opendlm_lock *lock = (struct opendlm_lock *)lk;
> +
> +	return lock->lvb;
> +}
> +
> +void opendlm_unhold_lvb(lm_lock_t * lk, lm_lvb_t * lvb)
> +{
> +	return;
> +}
> +
> +void opendlm_sync_lvb(lm_lock_t * lk, lm_lvb_t * lvb)
> +{
> +	return;
> +}
> +
> +void opendlm_reset_exp(lm_lockspace_t * lockspace, unsigned int jid)
> +{
> +	dlm_purge(jid+1, 0, 0);
> +}
> +
> +struct lm_lockops opendlm_ops = {
> +	proto_name:		PROTO_NAME,
> +	mount:			opendlm_mount,
> +	others_may_mount:	opendlm_others_may_mount,
> +	unmount:		opendlm_unmount,
> +	get_lock:		opendlm_get_lock,
> +	put_lock:		opendlm_put_lock,
> +	lock:			opendlm_lock,
> +	unlock:			opendlm_unlock,
> +	reset:			opendlm_reset,
> +	cancel:			opendlm_cancel,
> +	hold_lvb:		opendlm_hold_lvb,
> +	unhold_lvb:		opendlm_unhold_lvb,
> +	sync_lvb:		opendlm_sync_lvb,
> +	reset_exp:		opendlm_reset_exp,
> +};
> +
> +
> +int opendlm_start(void)
> +{
> +	return lm_register_proto(&opendlm_ops);
> +}
> +
> +void opendlm_stop(void)
> +{
> +	return lm_unregister_proto(&opendlm_ops);
> +}
> +
> +module_init(opendlm_start);
> +module_exit(opendlm_stop);
> +
> +MODULE_LICENSE("GPL");
> diff -Naur -X /usr/src/dontdiff
> opengfs-old/src/locking/modules/ogfs_dlm/Makefile.am
> opengfs-bak/src/locking/modules/ogfs_dlm/Makefile.am
> --- opengfs-old/src/locking/modules/ogfs_dlm/Makefile.am	1970-01-01
> 08:00:00.000000000 +0800
> +++ opengfs-bak/src/locking/modules/ogfs_dlm/Makefile.am	2003-09-04
> 19:51:32.000000000 +0800
> @@ -0,0 +1,25 @@
> +# -*-Makefile-*-
> +
> +#
> +#    Copyright 2001 The OpenGFS Project
> +#
> +#    This is free software released under the GNU General Public
> License.
> +#    There is no warranty for this software.  See the file COPYING for
> +#    details.
> +#
> +#    See the file AUTHORS for a list of contributors.
> +#
> +
> +EXTRA_INCL = \
> +	-I$(top_srcdir)/src/fs -I$(top_srcdir)/src/pool \
> +	-I$(top_srcdir)/src/stomith/module \
> +	-I/where/opendlm/src/include
> +
> +include $(top_srcdir)/make/modules.mk
> +
> +module_PROGRAMS = ogfs_dlm.o
> +ogfs_dlm_o_SOURCES = dlm.c deadman.c misc.c cluster_ondisk.c
> cluster_config.h ogfs_dlm.h
> +
> +EXEEXT =
> +ogfs_dlm.o$(EXEEXT): $(ogfs_dlm_o_OBJECTS)
> +	$(KLD) -r $^ -o $@
> diff -Naur -X /usr/src/dontdiff
> opengfs-old/src/locking/modules/ogfs_dlm/misc.c
> opengfs-bak/src/locking/modules/ogfs_dlm/misc.c
> --- opengfs-old/src/locking/modules/ogfs_dlm/misc.c	1970-01-01
> 08:00:00.000000000 +0800
> +++ opengfs-bak/src/locking/modules/ogfs_dlm/misc.c	2003-09-04
> 19:48:03.000000000 +0800
> @@ -0,0 +1,358 @@
> +/* -*- linux-c -* */
> +
> +#include <linux/module.h>
> +#include <linux/version.h>
> +#include <linux/init.h>
> +#include <linux/kmod.h>
> +#include <linux/kernel.h>
> +#include <linux/slab.h>
> +#include <linux/fs.h>
> +#include <linux/sched.h>
> +#include <linux/list.h>
> +#include <linux/kdev_t.h>
> +
> +#include <lm_interface.h>
> +#include <cluster_config.h>
> +#include "ogfs_dlm.h"
> +
> +struct ogfs_dlm *find_dlm(uint32 namespace)
> +{
> +	struct ogfs_dlm *tmp;
> +	
> +	list_for_each_entry(tmp, &dlm_list, list) {
> +		if (tmp->name_space == namespace) {
> +			return tmp;
> +		}
> +	}
> +	return NULL;
> +}
> +
> +
> +int setup_jid(struct ogfs_dlm *dlm, uint32 ip)
> +{
> +	struct buffer_head *bh;
> +	cluster_global_t cg;
> +	cluster_node_t cn;
> +	int block = 0, found = 0;
> +	kdev_t dev;
> +
> +	dev = dlm->cfile->f_dentry->d_inode->i_rdev;
> +	set_blocksize(dev, CIDEV_BLOCKSIZE);
> +
> +	bh = bread(dev, 0, CIDEV_BLOCKSIZE);
> +	if (!bh) {
> +		printk("%s:  can't read Cluster Information Device\n",
> +		       PROTO_NAME);
> +		return (-1);
> +	}
> +
> +	cluster_global_in(&cg, bh->b_data);
> +
> +	if (cg.cg_magic != OGFS_CLUSTER_GLOBAL) {
> +		mark_buffer_uptodate(bh, FALSE);
> +		brelse(bh);
> +		return (-1);
> +	}
> +
> +	if ((((cg.cg_version >> 16) & 0xff) !=
> +	     ((OGFS_CLUSTER_VERSION >> 16) & 0xff))
> +	    || (((cg.cg_version >> 8) & 0xff) !=
> +		((OGFS_CLUSTER_VERSION >> 8) & 0xff))) {
> +		mark_buffer_uptodate(bh, FALSE);
> +		brelse(bh);
> +		printk
> +		    ("%s:  version mismatch: cidev (%u, %u), memexp (%u, %u)\n",
> +		     PROTO_NAME, (cg.cg_version >> 16) & 0xff,
> +		     (cg.cg_version >> 8) & 0xff,
> +		     (OGFS_CLUSTER_VERSION >> 16) & 0xff,
> +		     (OGFS_CLUSTER_VERSION >> 8) & 0xff);
> +		return (-1);
> +	}
> +
> +	mark_buffer_uptodate(bh, FALSE);
> +	brelse(bh);
> +
> +	block++;
> +
> +	/*  Read cluster info  */
> +
> +	while (1) {
> +		bh = bread(dev, block, CIDEV_BLOCKSIZE);
> +
> +		if (!bh) {
> +			printk
> +			    ("%s:  error reading Cluster Information Device.\n",
> +			     PROTO_NAME);
> +			goto fail;
> +		}
> +
> +		cluster_node_in(&cn, bh->b_data);
> +
> +		if (cn.cn_magic == OGFS_CLUSTER_GLOBAL) {	/* end of the configured
> nodes */
> +			mark_buffer_uptodate(bh, FALSE);
> +			brelse(bh);
> +			break;
> +		}
> +
> +		if (cn.cn_magic != OGFS_CLUSTER_NODE)	/* a node was removed from the
> config */
> +			goto next;
> +
> +		if (cn.cn_ipaddr == ip) {	/* our own config block */
> +			dlm->jid = block - 1;
> +			found = TRUE;
> +		}
> +
> +	      next:
> +		mark_buffer_uptodate(bh, FALSE);
> +		brelse(bh);
> +		block++;
> +	}
> +
> +	if (!found) {
> +		char ipstr[16];
> +		inet_ntoa(ip, ipstr);
> +		printk("%s:  no cluster info for %s\n", PROTO_NAME, ipstr);
> +		goto fail_2;
> +	}
> +
> +	dlm->nodes_count = block - 1;
> + 
> +	return (0);
> +
> +      fail:
> +
> +	if (bh) {
> +		mark_buffer_uptodate(bh, FALSE);
> +		brelse(bh);
> +	}
> +
> +      fail_2:
> +
> +	return (-1);
> +}
> +
> +int grab_mount_lock(struct ogfs_dlm *dlm)
> +{
> +	struct opendlm_lock *lock;
> +	dlm_stats_t status;
> +
> +	lock = (struct opendlm_lock *)kmalloc(sizeof(struct opendlm_lock),
> GFP_KERNEL);
> +
> +	if(!lock)
> +		return -1;
> +
> +	memset(lock, 0, sizeof(struct opendlm_lock));
> +
> +	strncpy(lock->lockname.magic, OGFS_DLM_MAGIC, OGFS_DLM_MAGICLEN);
> +	lock->lockname.name_space = dlm->name_space;
> +	lock->lockname.lock_type = LM_TYPE_MOUNT;
> +	lock->lockname.lock_num = 0;
> +	lock->dlm = dlm;
> +
> +	status = my_dlmlock_sync(LKM_EXMODE, &lock->lksb, 0, lock,
> ignore_bast);
> +
> +	if(status != DLM_NORMAL)
> +		return -1; 
> +
> +	if(lock->lksb.status != DLM_NORMAL)
> +		return -1;
> +
> +	dlm->mount_lock = lock;
> +
> +	return 0;
> +}
> +
> +void release_mount_lock(struct ogfs_dlm *dlm)
> +{
> +	dlmunlock(dlm->mount_lock->lksb.lockid, NULL, LKM_FORCE,
> ignore_unlockast, NULL, NULL);
> +	kfree(dlm->mount_lock);
> +}
> +
> +void parse_lockreq(unsigned int cur_state, unsigned int req_state, int
> *lockmode, int *lockflags)
> +{
> +	if(cur_state != LM_ST_UNLOCKED) {
> +		*lockflags |= LKM_CONVERT;
> +	}
> +	
> +	switch(req_state){
> +	case LM_ST_EXCLUSIVE:
> +		*lockmode = LKM_EXMODE;
> +		break;
> +	case LM_ST_SHARED:
> +		*lockmode = LKM_PRMODE;
> +		break;
> +	case LM_ST_DEFERRED:
> +                *lockmode = LKM_CWMODE;
> +                break;
> +	default:
> +		printk("%s:  Invalid lock state %d\n",
> +                       PROTO_NAME, req_state);
> +	}
> +}
> +
> +/**
> + * atoi
> + *
> + * @c:
> + *
> + */
> +

I thought there was one of these in the kernel already

> +int
> +atoi(char *c)
> +{
> +	int x = 0;
> +
> +	while ('0' <= *c && *c <= '9') {
> +		x = x * 10 + (*c - '0');
> +		c++;
> +	}
> +
> +	return (x);
> +}
> +
> +/**                                                                                                                                            
* inet_aton
> + *
> + * @ascii: 
> + * @ip:
> +
> *                                                                                                                                             
*/

same here, and the rest of these

> +
> +int inet_aton(char *ascii, uint32 * ip)
> +{
> +	uint32 value;
> +	int x;
> +
> +	*ip = 0;
> +
> +	for (x = 0; x < 4; x++) {
> +		value = atoi(ascii);
> +		if (value > 255)
> +			return (-1);
> +		*ip = (*ip << 8) | value;
> +		if (x != 3) {
> +			ascii = strstr(ascii, ".");
> +		if (!ascii)
> +			return (-1);
> +		ascii++;
> +		}
> +	}
> +	return (0);
> +}
> +
> +
> +
> +/**                                                                                                                                           
> + *
> inet_ntoa                                                                                                                                  
> +
> *                                                                                                                                            
> + *
> @ascii:                                                                                                                                    
> + *
> @ip:                                                                                                                                       
> +
> *                                                                                                                                            
> + */
> +
> +void inet_ntoa(uint32 ip, char *buf)
> +{
> +	int i;
> +	char *p;
> +
> +	p = buf;
> +	for (i = 3; i >= 0; i--) {
> +		p += sprintf(p, "%d", (ip >> (8 * i)) & 0xFF);
> +	if (i > 0)
> +		*(p++) = '.';
> +	}
> +	
> +}
> +
> +void lock_ast(void *astarg)
> +{
> +	struct opendlm_lock *lock = (struct opendlm_lock *)astarg;
> +
> +	lock->done = 1;
> +}
> +
> +
> +dlm_stats_t my_dlmlock_sync(int mode, struct lockstatus *lksb, int
> flags,
> +			    struct opendlm_lock *lock, dlm_bastlockfunc_t * bast)
> +{
> +	dlm_stats_t status;
> +
> +	status = dlmlock(mode, lksb, flags, (void *)(&lock->lockname),
> +				 sizeof(struct opendlm_lockname), lock_ast,
> +				(void *)lock, bast);
> +
> +	if(status != DLM_NORMAL)
> +		goto out; 
> +
> + sleep:
> +	if(!lock->done && !lock->cancel) {
> +		current->policy |= SCHED_YIELD;
> +		schedule();
> +		goto sleep;
> +	}
> +
> + out:
> +	return status;
> +}
> +
> +
> +
> +
> +void block_others(void *bastargs, int mode)
> +{
> +	struct opendlm_lock *lock = (struct opendlm_lock *)bastargs;
> +	lm_lockname_t name;
> +	
> +	name.ln_type = lock->lockname.lock_type;
> +	name.ln_number = lock->lockname.lock_num;
> +	
> +	switch(mode) {
> +	case LKM_EXMODE:
> +		lock->dlm->cb(lock->dlm->fsdata, LM_CB_NEED_E, (void *)&name);
> +		break;
> +
> +	case LKM_PRMODE:
> +		lock->dlm->cb(lock->dlm->fsdata, LM_CB_NEED_S, (void *)&name);
> +		break;
> +
> +	case LKM_CWMODE:
> +		lock->dlm->cb(lock->dlm->fsdata, LM_CB_NEED_D, (void *)&name);
> +		break;
> +		
> +	default:
> +		printk("%s:  Unkonwn lock mode %d!\n", PROTO_NAME, mode);
> +	}
> +	
> +	return;
> +}
> + 
> +
> +#include <crc.h>

move to the top

> +
> +/**
> + * locktable_hash - hash an array of data
> + * @data: the data to be hashed
> + * @len: the length of data to be hashed
> + *
> + * Take some data and convert it to a 32-bit hash.
> + *
> + * The hash function is a 32-bit CRC of the data.  The algorithm uses
> + * the crc_32_tab table above.
> + *
> + * This may not be the fastest hash function, but it does a fair bit
> better
> + * at providing uniform results than the others I've looked at.  That's
> + * really important for efficient directories.
> + *
> + * Returns: the hash
> + */
> +
> +uint32
> +locktable_hash(const char *data, int len)
> +{
> +	uint32 hash = 0xFFFFFFFF;
> +
> +	for (; len--; data++)
> +		hash = crc_32_tab[(hash ^ *data) & 0xFF] ^ (hash >> 8);
> +
> +	hash = ~hash;
> +
> +	return (hash);
> +}
> diff -Naur -X /usr/src/dontdiff
> opengfs-old/src/locking/modules/ogfs_dlm/ogfs_dlm.h
> opengfs-bak/src/locking/modules/ogfs_dlm/ogfs_dlm.h
> --- opengfs-old/src/locking/modules/ogfs_dlm/ogfs_dlm.h	1970-01-01
> 08:00:00.000000000 +0800
> +++ opengfs-bak/src/locking/modules/ogfs_dlm/ogfs_dlm.h	2003-09-04
> 16:26:18.000000000 +0800
> @@ -0,0 +1,63 @@
> +/* -*- linux-c -*- */
> +
> +#include <dlm.h>
> +
> +#define PROTO_NAME "opendlm"
> +
> +struct opendlm_lock;
> +
> +struct ogfs_dlm {
> +	struct list_head	list;
> +	char			table_name[256];
> +	uint32			name_space; /* hash value of table_name */
> +	lm_callback_t		cb;
> +	lm_fsdata_t		*fsdata;
> +
> +	struct file		*cfile;
> +	unsigned int		jid;
> +	unsigned int		nodes_count;
> +	struct opendlm_lock	*mount_lock;
> +	struct list_head	deadman;
> +};
> +
> +#define OGFS_DLM_MAGICLEN	4
> +#define OGFS_DLM_MAGIC		"OGFS"
> +
> +struct opendlm_lockname {
> +	char			magic[OGFS_DLM_MAGICLEN];
> +	uint32			name_space;
> +	uint64			lock_num;
> +	unsigned int		lock_type;
> +}__attribute__((packed));
> +
> +struct opendlm_lock {
> +	struct opendlm_lockname	lockname;
> +	struct lockstatus	lksb;
> +	lm_lvb_t		*lvb;
> +	struct ogfs_dlm		*dlm;
> +	int			cancel;
> +	int			done;
> +};
> +
> +extern struct list_head dlm_list;
> +
> +/* function defined in deadman.c */
> +void ignore_bast(void *bastargs, int mode);
> +void ignore_ast(void *bastargs);
> +void ignore_unlockast(void *unlockast, int lstat, void *extrap);
> +int start_deadman_lock(struct ogfs_dlm *dlm, unsigned int *first);
> +int stop_deadman_lock(struct ogfs_dlm *dlm);
> +
> +/* function defined in misc.c */
> +struct ogfs_dlm* find_dlm(uint32 namespace);
> +int setup_jid(struct ogfs_dlm *dlm, uint32 ip);
> +int grab_mount_lock(struct ogfs_dlm *dlm);
> +void release_mount_lock(struct ogfs_dlm *dlm);
> +void parse_lockreq(unsigned int cur_state, unsigned int req_state, int
> *lockmode, int *lockflags);
> +uint32 locktable_hash(const char *data, int len);
> +void block_others(void *bastargs, int mode);
> +int inet_aton(char *ascii, uint32 * ip);
> +void inet_ntoa(uint32 ip, char *buf);
> +dlm_stats_t my_dlmlock_sync(int mode, struct lockstatus *lksb, int
> flags, 
> +		struct opendlm_lock *lock, dlm_bastlockfunc_t * bast);
> +
> 
> 

It looks good at  first glance, I do think the naming should be given a little 
more thought. The other modules are named (and in directories) called memexp, 
nolock, etc. I don't think the name ogfs_dlm is true to that standard (even 
though it mainly us that work on opendlm). Like I said before, if we stick to 
the the current naming convention (i.e. if this directoy is named opendlm) 
then it will cause less confusion down the road if we try to work with 
another dlm instead or in addition to opendlm.

--Brian Jackson


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Opengfs-devel mailing list
Opengfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opengfs-devel

[Kernel]     [Security]     [Bugtraq]     [Photo]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Clusters]     [Linux RAID]     [Yosemite Hiking]     [Linux Resources]

Powered by Linux