Re: Custom VFS

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]



Hi Daniel,

Just a note of encouragement...
I have so far written 2 filesystems in Java that use Samba for 2 different companies, so you're not alone! :-)

The strategy I've used is to write a simple TCP protocol client (the VFS module) and server (a straight forward threaded Java server). Works like a charm. As long as the client side is abstracted enough so that its samba connection state is independent from the server connection state, there are no issues with restarting either. (I started out using a statefull protocol, but ended up changing to a completely stateless one, where the individual messages contain enough information to establish context. This way, if either end of the system goes down, recovery is the simple act of building a new TCP connection.)

I also attempted to use the Apache ActiveMQ C++ library for communication, but found it buggy and leaky.

I originally looked into hosting the JVM in the VFS module, but that was going to be a problem because each smbd process would have to start its own JVM. The JVM startup time (especially the server JVM) is very high and the memory overhead would not make it scalable.

TCP through the loopback interface is very fast (at least on the linux system's I've developed for), so there was no need to implement some sort of shared memory interface.

The system I'm working on now manages PB class storage (currently up to 10PB) with hundreds of concurrent clients and the VFS module does this without issues or much overhead. We're regularly seeing write speeds in the 400-500MB/s range using 10GbE and multiple windows clients.

Good luck!

P.S.: Blatant plug for my current project: http://www.cuttedge.com/psca/index.html

On 07/24/2012 01:15 PM, ivenhov wrote:
Thanks Michael, great link.
Exactly what I was looking for. Does not answer my JNI question (creating
file system in Java is not that common),
but it's a great starting point.

Daniel

On 24 July 2012 19:51, Michael Wood-8 [via Samba] <
ml-node+s2283325n4634960h22@xxxxxxxxxxxxx> wrote:

Hi

On 19 July 2012 11:39, Daniel Iwan <[hidden email]<http://user/SendEmail.jtp?type=node&node=4634960&i=0>>
wrote:

Hi

I would like to  create custom VFS that would redirect all calls to my
backend.

Few questions:

1) is it possible to use Java with JNI wrapper to communicate with my
backend or does it have to be pure C/C++ ?
As far as I know, a Samba VFS must be a shared library.  i.e. a .so
file, so I don't think Java would work (although I don't know much
about JNI.)

2) are the notifications (file deletion/creation etc.) available in VFS,
in
other words notifying samba server and applications that directory has
changed?

3) are there any examples/tutorials/walkthroughs or up to date
documentation for VFS?
Try this:

http://www.samba.org/~sharpe/The-Samba-VFS.pdf

--
Michael Wood <[hidden email]<http://user/SendEmail.jtp?type=node&node=4634960&i=1>>

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


------------------------------
  If you reply to this email, your message will be added to the discussion
below:
http://samba.2283325.n4.nabble.com/Custom-VFS-tp4634738p4634960.html
  To unsubscribe from Custom VFS, click here<http://samba.2283325.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4634738&code=aXdhbi5kYW5pZWxAZ21haWwuY29tfDQ2MzQ3Mzh8LTYxNzE4NjQwNw==>
.
NAML<http://samba.2283325.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>




--
View this message in context: http://samba.2283325.n4.nabble.com/Custom-VFS-tp4634738p4634963.html
Sent from the Samba - General mailing list archive at Nabble.com.

--
Andrew Scherpbier
andrew@xxxxxxxxxxxxxx

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


[Linux]     [Info Cyrus]     [LARTC]     [Bugtraq]     [Netfilter]     [Internet Dating Forums]     [RAID]     [Yosemite News]     [Photography]

Add to Google Powered by Linux