|[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]|
> For someone working above the Datamover the plain meaning of "guarantee > that all the necessary data transfers take place" is that the Datamover > will > reliably deliver the data in the absence of transport errors.
> It should be made clear that a user of a Datamover service is responsible > for avoiding this problem.
>In my opinion the absence of enabling language > would actually forbid some of the solutions you listed.
> It also assumes that a Datamover is a *distributed* solution. What if > the protocol being implemented in a Datamover architecture was merely > a form of inter-process communication or use of shared memory?
Sorry, what if so?
>Requiring Datamovers using very different protocols > to mimic that behavior is wrong
>iSCSI is already 99% flow controlled, > clearly labeling that it is 100% responsible should not > be a problem.
Caitlin Bestler wrote:
On Sep 13, 2004, at 9:23 PM, Mallikarjun C. wrote:
Caitlin Bestler wrote:
What is the responsibility for the distribution of flow control between the Data Mover components and the iSCSI peers?
Datamover Architecture doesn't prescribe a specific distribution, it only sets functionality expectations.
The position of the Architecture doc is that it's upto individual Datamover protocols (e.g., iSER) to require a specific distribution.
Section 8 specifies: 1) guarantee that all the necessary data transfers take place when the local iSCSI layer requests transmitting a command (in order to complete a SCSI command, for an initiator),or sending/receiving an iSCSI data sequence (in order to complete part of a SCSI command, for a target).
This functional requirement can be met by a Datamover protocol in one of a few ways:
1. Must break the underlying transport connection when this cannot be met.
2. Must build in a positive flow control protocol internal to itself.
3. Must place additional requirements on iSCSI-3720's behavior (e.g., "shall only send a
max of 'n' immediate commands" etc.)
4. Must take the help of *its* LLPs (e.g., an iWARP stack for iSER) to realize the same functionality.
DA does not have a prescription for iSER here.
After seriously evaluating #2, iSER chose a combination of #1 and #4 (#4: recall the lengthy discussion
on DDP text wrt "graceful handling" of empty RQ exceptions).
Note that despite its best intentions, a Datamover protocol is not changing the iSCSI-3720's behavior -
e.g. too many immediate commands can be thrown away by iSCSI, without notice.
I do not see an interoperability problem here for iSER, nor am convinced that DA should say more
than it currently does in this regard.
For someone working above the Datamover the plain meaning of "guarantee
that all the necessary data transfers take place" is that the Datamover will
reliably deliver the data in the absence of transport errors.
Option one is indeed an excellent remedy when the Datamover consumer has sent more iSCSI PDUs than its peer can consume. But a Datamover consumer may have gotten into the habit of assuming that the transport layer will sweep pacing problems under the rug of its own pacing. TCP applications frequently place more messages in flight than could be instantaneously received on the other end, relying on transport layer buffering to smooth out the problem.
But the primary wire protocol that the Datamover architecture was designed to support, iSER, does not offer transport layer buffering because RDMA is designed to eliminate transport layer buffering.
It should be made clear that a user of a Datamover service is responsible for avoiding this problem. In my opinion the absence of enabling language would actually forbid some of the solutions you listed.
Inviting traffic onto the network that you do not have buffers to handle is bad enough. But solving it through network layer retransmission is clearly an incorrect solution.
It also assumes that a Datamover is a *distributed* solution. What if the protocol being implemented in a Datamover architecture was merely a form of inter-process communication or use of shared memory?
Leaving the options open to each implementation is perfectly fine, but only if those options are made clear to Datamover users. Since Datamover is replacing carrying iSCSI PDUs over TCP then there will be a natural tendency to assume that the solutions will match those employed by TCP. Since those are not the optimal solutions it is more likely that a buffer overrun will result in a connection being dropped even when there was no network error.
That is actually acceptable, if it is a response to sending excessive iSCSI PDUs. The fact that TCP would have been forgiving does not make them legitimate somehow.
The fundamental fact is that the options I outlined *are* the only options. TCP uses transport flow control and transport buffering, and when necessary blocks the transmitter from submitting more requests. Requiring Datamovers using very different protocols to mimic that behavior is wrong -- but that is the plain meaning of the requirements as currently stated.
It is not my intention to require iSER to add its own flow control. I am trying to make certain that the flow control problem is not shoved down to the RDMA layer and thereby prevent implementations from truly eliminating TCP-caused bottlenecks. That requires moving responsibility for flow control clearly back up to the application layer. iSCSI is already 99% flow controlled, clearly labeling that it is 100% responsible should not be a problem.
_______________________________________________ Ips mailing list Ips@ietf.org https://www1.ietf.org/mailman/listinfo/ips