One document matched: draft-ietf-ips-fcovertcpip-00.txt
IPS Working Group M. Rajagopal, R. Bhagwat, LightSand Communications
INTERNET-DRAFT E. Rodriguez, Lucent Technologies
<draft-ietf-ips-fcovertcpip-00.txt> V. Chau, Gadzoox Networks
(Expires March 31, 2001) S. Berman, Vixel
S. Wilson, Brocade Communications
M. O'Donnell, McDATA
C. Carlson, QLogic
Fibre Channel Over TCP/IP (FCIP)
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC 2026 [1].
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
monthsx and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-Drafts
as Reference material or to cite them other than as ``work in
progress''.
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/lid-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
1. Abstract
Fibre Channel (FC) is a dominant technology used in Storage Area
Networks (SAN). The purpose of this draft is to specify a standard
standard way of encapsulating FC frames over TCP/IP and to describe
mechanisms that allow islands of FC SANs to be interconnected over
IP-based networks. FC over TCP/IP relies on IP-based network
services to provide the connectivity between the SAN islands over
LANs, MANs, or WANs. The FC over TCP/IP specification relies upon
TCP for congestion control and management and upon both TCP and FC
for data error and data loss recovery. FC over TCP/IP treats all
classes of FC frames the same -- as datagrams.
2. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this document are to be interpreted as described in RFC 2119 [2].
Rajagopal, et al. [Page 1]
Internet-Draft Fibre Channel over TCP/IP October 2000
3. Motivation and Objectives
Fibre Channel (FC) is a gigabit speed networking technology
primarily used for Storage Area Networking (SAN). Fibre Channel is
standardized under American National Standard for Information
Systems of the National Committee for Information Technology
Standards (ANSI-NCITS) and has specified a number of documents
describing its protocols, operations, and services [13].
The motivation behind connecting remote sites include disk or tape
backup and live mirroring, or simply distance extension between two
FC devices or FC Switch clusters (SAN islands).
A fundamental assumption made in this specification is that the
Fibre Channel traffic is carried over the IP network in such a
manner that the Fibre Channel fabric and all Fibre Channel devices
on the fabric are unaware of the fact. This means that the FC
datagrams must be delivered in such time as to comply with
existing Fibre Channel specifications. The FC traffic may span
LANs, MANs and WANs, so long as this fundamental assumption is
adhered to.
Fibre Channel operates at Gigabit speeds. This specification
is written such that Fibre Channel traffic may be transported over
an IP backbone that has been engineered to have equivalent or
better bit-error-rate (BER) and line speed as the Fibre Channel
environments being bridged. While the tunneling of Fibre Channel
traffic over other IP networks not so engineered is not precluded,
the above environment is an important one, and this specification
has been written so that to optimize for such traffic, while not
over-burdening other configured IP networks.
The main objectives of this document are to:
1) specify the TCP encapsulation and mapping of FC frames
2) apply the mechanism described in (1) to an FC network using
IP as a backbone, or more generally, between any two FC
devices.
3) address any FC concerns arising from tunneling FC traffic
over an IP network, including security, data integrity
(loss), congestion, and performance. This will be
accomplished, where appropriate, by utilizing the existing
IETF-specified suite of protocols.
4) be backwards compatible with existing FC specifications.
While new work may be undertaken in T11[13] to optimize
and enhance the bridging of FC networks and fabrics, this
specification will not require adherence to such future
works.
Rajagopal, et al. [Page 2]
Internet-Draft Fibre Channel over TCP/IP October 2000
4. FCIP Protocol
4.1 FCIP Device
In this specification, the term FCIP device generally refers to
any device that encapsulates FC frames into TCP segments and re-
assembles TCP segments to regenerate FC frames.
Note: In an actual implementation, the FCIP device may be a stand-
alone box or integrated with an FC device such as an FC back-
bone switch (BBW) or integrated with any TCP/IP device such
as an IP switch or an IP router.
The FCIP device is a transparent translation point. The IP network
is not aware of the FC payload that it is carrying. Likewise, the
FC fabric and the FC end nodes are unaware of the IP-based
transport.
4.2 Protocol
The FCIP protocol specifies the TCP/IP encapsulation, mapping and
routing of FC frames and applies these mechanisms to an FC network
utilizing IP for its backbone, or more generally, between any two
FC devices. The FCIP protocol is summarized below:
1. All FCIP protocol devices are peers and communicate using
TCP/IP. Each FCIP device behaves like a TCP end-node from the
perspective of the IP-based network. That is, these devices do
not perform IP routing or IP switching but simply forward FC
frames.
2. There is no requirement for an FCIP device to establish a
login with a peer before communication begins. However, FCIP
devices may authenticate the IP packet before accepting it
using the IPSec protocols. Each IP datagram is treated
independently and an FCIP device receiver simply listens to
the appropriate socket value contained in the TCP header.
3. Each FCIP device may be statically or dynamically configured
with a list of IP addresses corresponding to all the
participating FCIP devices. Dynamic discovery of participating
FCIP devices may be performed using Internet protocols such as
LDAP, DHCP or other discovery protocols. It is outside the
scope of this specification to describe any static or dynamic
scheme for participating FCIP device IP address discovery.
4. Discovery of FC addresses (accessible via the FCIP device)
is provided by techniques and protocols within the FC
architecture. These techniques and protocols are described in
Fibre Channel ANSI standards ([3], [7], [15]). The FCIP device
does not participate in the discovery of FC addresses. Routing
in the IP plane and the FC plane are largely independent.
Rajagopal, et al. [Page 3]
Internet-Draft Fibre Channel over TCP/IP October 2000
5. The exact path (route) taken by an FC over TCP/IP encapsulated
packet follows the normal procedures of routing any IP packet.
From the perspective of the FCIP devices this communication is
between only two FCIP devices for any given packet.
6. An FCIP device may send FC encapsulated TCP/IP packets to more
than one FCIP device. However, these encapsulated packets are
treated as separate instances and are not correlated in any
way by the FCIP protocol devices. The source FCIP device
routes its packets based on the 3-byte FC destination Address
Identifier (D_ID) contained in each FC frame.
7. An IP packet may make use of the IPSec protocol to provide
secure communications across the IP-based network.
8. Any re-ordering of data link frames due to MTU fragmentation
will be recovered in accordance with a normal TCP reliable
delivery behavior.
Any re-ordering of FC frames due to IP packet re-ordering will
be resolved via the standard TCP reliable delivery behavior.
9. FCIP relies on both TCP error recovery mechanism and normal FC
recovery mechanisms to detect and recover from data loss due
to any loss of IP packets.
10. FC over TCP/IP encapsulated IP packets shall indicate the use
of the Premium Service in the DSCP bits in the IP header.
11. The TCP layer in the sending FCIP device shall package each
FC frame handed down by the FC layer into a TCP segment and
set the PSH control flag in the TCP header to ensure that the
entire FC frame is sent in one TCP segment. If the FC frame
cannot be packaged in one TCP segment (e.g. the FC frame size
is greater than TCP MSS), the last part of the FC frame must
occupy one TCP segment and the PSH of that segment must be
set.
5. FCIP Encapsulation
5.1 FC Frame Format
All FC frames have a standard format much like LAN's 802.x
protocols. However, the exact size of each frame varies depending
on the sizes of the variable fields. The sizes of the variable
fields range from 0 to 2112-bytes as shown in the FC Frame Format
in Fig. 1 resulting in the minimum size FC Frame of 36 bytes and
the maximum size FC frame of 2148 bytes.
Rajagopal, et al. [Page 4]
Internet-Draft Fibre Channel over TCP/IP October 2000
+------+--------+-----------+----//-------+------+------+
| SOF |Frame |Optional | Frame | CRC | EOF |
| (4B) |Header |Header | Payload | (4B) | (4B) |
| |(24B) |<----------------------->| | |
| | | Data Field = (0-2112B) | | |
+------+--------+-----------+----//-------+------+------+
Fig. 1 FC Frame Format
Start Of Frame (SOF) and End Of Frame (EOF) Delimiters:
On an FC link, Start-of-Frame (SOF) and End-Of-Frame (EOF) are
called Ordered Sets and are sent as special words constructed
from the 10-bit comma character (K28.5) followed by three
additional 10-bit data characters. On a non-Fibre Channel link,
the SOF and EOF delimiters are both byte-encoded and 4-bytes
long.
On an FC link the SOF delimiter serves to identify the beginning
of a frame and prepares the receiver for frame reception. The SOF
contains information about the frame's Class of Service, position
within a sequence, and in some cases, connection status.
The EOF delimiter identifies the end of the frame and the final
frame of a sequence. In addition, it serves to force the running
disparity to negative. The EOF is used to end the connection in
connection-oriented classes of service.
It is therefore important to preserve the information conveyed by
the delimiters across the IP-based network, so that the receiving
FCIP device can correctly reconstruct the FC frame in its
original SOF and EOF format before forwarding it to its ultimate
FC destination on the FC link.
Start of Frame (SOF) and End of Frame (EOF) byte-encodings are
defined in Annex A. Although the SOF and EOF codes are 32-bits,
the format makes use of a single-byte to represent each FC
Ordered Set. The remaining three bytes associated with these
delimiters in the TCP encapsulated FC frames do not necessarily
carry any meaningful information.
Frame Header:
The Frame Header is 24-bytes long and has several fields that are
associated with the identification and control of the payload.
Current FC Standards allow up to three Optional Header fields
[4],[5]:
- Network_Header (16-bytes)
- Association_Header (32-bytes)
- Device_Header (up to 64-bytes).
Rajagopal, et al. [Page 5]
Internet-Draft Fibre Channel over TCP/IP October 2000
Frame Payload:
The FC Frame Payload is transparent to the FCIP device. An
FC application level payload is called an Information Unit at the
FC-4 Level. This is mapped into the Frame Payload of the FC
Frame. A large Information Unit is segmented using a structure
consisting of FC Sequences. Typically, a Sequence consists of
more than one FC frames. FCIP does not maintain any state
information regarding the relationship of frames within an FC
Sequence.
CRC:
The CRC is 4-bytes long and uses the same 32-bit polynomial used
in FDDI and is specified in ANSI X3.139 Fiber Distributed Data
Interface.
Note: When FC frames are encapsulated into TCP segments, the CRC
is untouched.
5.2 FC Frame Mapping to TCP/IP packets
Fig.2 shows the mapping of the FC frame into an TCP/IPv4 Packet.
The FC to TCP/IP mapping (and the reverse) mapping is one-to-one
since the maximum size of the encapsulated FC Frame along with
the header fields does not exceed 2148 bytes.
The minimum size FC Frame is 36 bytes resulting in a maximally
minimum IP MTU size of 156 bytes. (The maximally minimum MTU size
is the IP packet with the minimum size payload and the maximum
size IP headers and TCP headers).
The maximum size FC frame is 2148 bytes resulting in a nominal IP
packet size of 2188 bytes. Fig. 2 shows the format of the IPv4
packet with the standard 20-byte IP fixed header, a 40-byte
optional IP header, the standard 20-byte TCP fixed header and TCP
options. For the case of the maximum size payload of 2148 bytes,
the maximum TCP/IPv4 packet size is 2268 bytes.
The maximum size FC frame can cause the IP packet to be
fragmented when the data link cannot support this MTU size.
Furthermore, TCP may also segment the data if the maximum segment
size (MSS) is less than the size of the frame. When an IP packet
is fragmented, the required parts of the header must be copied by
all fragments and the option field may or may not be copied. When
a TCP packet is segmented, normal TCP operations are responsible
for receiving and re-sequencing the data prior to passing it on
to the FC processing portion of the device.
Rajagopal, et al. [Page 6]
Internet-Draft Fibre Channel over TCP/IP October 2000
+-----------+--------------+----------+-----------+-----------+
| IP Header |IP Opt. Header|TCP Header|TCP Options| FC Frame |
|(20 bytes) | (40 bytes |(20 bytes)| (40 bytes |(2148 bytes|
| | Max) | | Max) | Max) |
+-----------+--------------+----------+-----------+-----------+
Fig. 2 Format of an IPv4 Payload carrying FC
<Placeholder for IPv6 format>
If IPSec is used for security it introduces its own headers and
the IP packet size increase depends on the exact mode of IPSec
usage (AH versus ESP, Tunnel versus Transport). However, this
additional increase in the IP packet size due to IPSec headers is
relatively small (see [8], [9], [10]), and the maximum size IP
packet will remain within its maximum size of 65535 bytes. Adding
IPSec header may, in some cases, lead to fragmentation if it
exceeds the data link MTU.
IP Header Field Setting:
DSCP (6 bits): The Differentiated Service Code Points (DSCP) [6]
shall be set to correspond to the Premium Service. This service
provides "Expedited Forwarding" at each IP hop (Per Hop Behavior
(PHB)).
Protocol (8 bits): This 8-bit field defines the higher level
protocol that uses the service of the IP layer. In this case,
this is set to the TCP Protocol Value 6 as defined in [12].
Source IP Address (32 bits): This is the IP address of the
ingress FCIP device that is transmitting the IP packet containing
the FC frame.
Destination IP Address (32 bits): This is the IP address of
the egress FCIP device that is receiving the IP packet containing
the FC frame.
The FCIP specification treats all classes of FC frames the same -
as datagrams. Since FC Primitives and Primitive Sequences are not
exchanged between FCIP devices there may be times when a frame
is lost in the IP network. When this event occurs it is the
responsibility of the communicating Fibre Channel devices to
detect and correct the errors. The FCIP devices shall not
generate Fibre Channel F_BSY and/or F_RJT frames or otherwise
participate in FC frame recovery. FCIP may not be suitable for
the transport of Class 1 traffic since these frames are treated
the same way as any Class 2 or 3 frame by the FCIP devices.
Rajagopal, et al. [Page 7]
Internet-Draft Fibre Channel over TCP/IP October 2000
5.3 FC frame mapping to TCP Segment
The FC-to-TCP mapping (and reverse mapping) may not necessarily
be one-to-one as the FC frame size may exceed the TCP Maximum
Segment Size (MSS). In this case, the TCP layer in the sending
FCIP device may segment the FC frame into multiple TCP segments.
The sending device must ensure that the last TCP segment contains
only the last part of the encapsulated FC frame and that last TCP
segment does not contain the beginning of another FC frame. The
last TCP segment must also have the PSH flag set so that the
receiving FCIP device knows to send the frame to the FC layer
above it. The fields in the TCP header are explained in the
following paragraphs:
Source Port:
The port number selected by the sending FCIP device.
Destination Port:
The well-known port number assigned for FC encapsulation. This
number is assigned by ... <to be added>
Sequence Number:
The sequence number of the SOF byte-code of the FC frame or the
first byte of the current TCP segment payload.
Acknowledge Number:
If the ACK flag is set, this field contains the sequence number
of the next byte that the receiving TCP expects to receive.
Data Offset:
The number of 32-bit words in the TCP header. The use of TCP
options are not precluded for FCIP devices.
Control (flag) bits:
URG: Urgent Pointer field significant - must be 0; the use
of urgent data is not allowed for segments containing
encapsulated FC frames.
ACK: Acknowledgment field significant - FCIP devices use
this bit to indicate that the Acknowledge number is
valid.
PSH: Push Function - The sending FCIP device shall set this
bit to 1 if the entire FC frame fits inside a TCP
Rajagopal, et al. [Page 8]
Internet-Draft Fibre Channel over TCP/IP October 2000
segment or the last portion of an FC frame (containing
the EOF) is sent in its own TCP segment.
RST: Reset the connection - the FCIP device shall set this
flag according to the state of the TCP connection and
in compliance to the TCP specifications.
SYN: Synchronize sequence numbers - the FCIP device shall
set this flag according to the state of the TCP
connection and in compliance to the TCP specifications.
FIN: No more data from sender - the FCIP device shall set
this flag according to the state of the TCP connection
and in compliance to the TCP specifications.
Window:
The FCIP device shall operate this field according to the state
of the TCP connection and in compliance to the TCP
specifications.
Checksum:
The FCIP device shall calculate the checksum over the entire FC
frame or the portion of the FC frame encapsulated inside the
current segment. The checksum shall also include the TCP header
and the pseudo IP header as specified in [16].
Urgent Pointer:
The FCIP devices shall not use this field as the urgent function
is not allowed in tunneling FC frames over the IP network.
Options and Padding:
FCIP devices are allowed to use TCP options and padding as
specified by the TCP specifications [16].
5.4 Fibre Channel Bit and Byte Ordering
Fibre Channel's FC-1 Level defines the method used to encode data
prior to transmission and subsequently decode the data upon
reception. The method encodes 8-bit bytes into 10-bit characters
to improve the transmission characteristics of the serial data
stream. In Fibre Channel, data fields are aligned on word
boundaries. A word in FC is defined as 4 bytes or 32 bits. The
resulting transmission word after the 8-bit to 10-bit encoding
consists of 40 bits.
Data words or Ordered Sets (special FC-2 Level control words)
from the FC-2 Level map to the FC-1 Level with no change in order
Rajagopal, et al. [Page 9]
Internet-Draft Fibre Channel over TCP/IP October 2000
and the bytes in the word are transmitted in the Most Significant
Byte first to Least Significant Byte order. The transmission
order of bits in each byte is from the Least Significant Bit to
the Most Significant Bit.
6. FCIP Network
6.1 FC Backbone Switches
Fibre Channel Standards [3] describe the operation of and
interactio betweenn FC Switches. Two distinct levels of switch
interconnections are specified. Autonomous Regions (AR) are
defined to allow clusters of FC Switches to be connected across a
backbone network called an FSPF-backbone. An Autonomous Region is
administratively defined with each AR encompassing one or more FC
Domains. The FSPF-backbone network is formed from one or more
Backbone Switches (BSW) that run the FSPF-backbone routing
protocol. The FSPF-backbone routing protocol is based on OSPF and
the FSPF backbone may consist of an arbitrary mesh network. A
BSW may communicate with multiple neighbors. As specified in [3],
native FC frames traverse the backbone between BSW neighbors. The
FSPF-backbone Routing Protocol messages are exchanged between
BSWs on the FSPF backbone.
An example network consisting of 4 ARs and an FSPF backbone
consisting of 3 links is given in Fig. 3. There is no restriction
in adding other links to this network as needed. The connection
between BSWs below may in fact form a fully connected mesh.
_______ _______
| | | |
| AR #1 |_____ _____| AR #4 |
|_______| | | |_______|
___|___ ___|___
| BSW 1 |---------------------| BSW 4 |
|_______| |_______|
___|___ _______
| BSW 2 |---------------------| BSW 3 |
|_______| |_______|
___ ___ | | _______
| | | | | |
| AR #2 |----- -----| AR #3 |
|_______| |_______|
Fig. 3 Example Network showing FSPF-Backbone Switching
Architecture
Note:
BSW 1 knows it is connected to BSWs 2 and 4;
BSW 2 knows it is connected to BSWs 1 and 3;
BSW 4 knows it is connected to BSWs 1.
Rajagopal, et al. [Page 10]
Internet-Draft Fibre Channel over TCP/IP October 2000
An FCIP device provides a single, logical interface to the FSPF-
backbone protocol connecting multiple BSW neighbors on the IP-
network. From the FSPF-backbone routing's point of view, the
connection to each neighbor on the IP-network is treated as a
separate logical FC link.
In FCIP, the native FC frames are first encapsulated in TCP
segments which then traverse the IP-based network. The IP network
provides a new transport path for each emulated FSPF-backbone
link.
The IP network itself may consist of any number of hops between
two FCIP devices. Also, the route taken by the IP packet between
any two FCIP devices is dictated by normal IP routing.
A functional and logical diagram of an IP-based FSPF-backbone for
the example network given in Fig. 3 is shown in Fig. 4. In this
figure, each BSW is logically connected to other BSWs.
_______ _______
| | | |
| AR #1 |--- | AR #4 |
|_______| | ______ ________ ______ |_______|
__|_ __ | | | | | | ___|___
| BSW 1 |---| FCIP |--| IP |--| FCIP |--| BSW 4 |
|_______| |______| | Network| |______| |_______|
| |
--------
______ | | ______
______ | | | | | | _______
| BSW 2|---| FCIP |-----| |---| FCIP |---| BSW 3 |
|______| |______| |______| |_______|
________ | ___|___
| | | | |
| AR #2 |__| | AR #3 |
|________| |_______|
Fig. 4 Example Network showing an IP-based FC Backbone Switching
Architecture
The IP-based network has transformed the FSPF-backbone into a
fully connected network. From the perspective of each BSW all
remote BSWs therefore appear to be neighbors. The FSPF-backbone
routing protocol computations would make the IP-based network
topology appear as a fully connected mesh.
The FSPF-backbone routing protocol exchanges between BSWs occur
transparently to the FCIP devices. Encapsulated FC frames are
routed on the IP network according to the normal IP routing
Rajagopal, et al. [Page 11]
Internet-Draft Fibre Channel over TCP/IP October 2000
procedures. In this mode, the FSPF-backbone routing protocol lays
over the IP network and has no knowledge of the underlying IP
protocol and IP routing or the underlying technology that carries
the IP datagram. This concept is shown in Fig.5
________ _______
| AR #1 | | AR #2 |
| |-- | |
|________| | ______ ________ _____ |_______|
__|___ | | | | | | ____|__
| BSW 1|---| FCIP |--| IP |--|FCIP |--| BSW 2 |
| | | | | Network| | | | |
|______| |______| |________| |_____| |_______|
<-------------->
IP Routing
<---------------------------------->
FSPF-backbone Routing Plane
Fig. 5 FC packet routing over IP based backbone network
Note: IP Network routing may consist of multiple paths
6.2 FC Device
The protocol encapsulation and mapping of the FC frame described
in earlier sections applies equally to any pair of FC devices
(e.g. switch to switch or host to storage subsystem) wishing to
tunnel FC frames across an IP-based network. Any FC routing
protocol exchanges may still occur transparent to the FCIP
devices. It should be noted that Fibre Channel Primitive
Sequences and Primitives are not exchanged between FCIP devices.
7. Security Considerations
Using a wide-area, general purpose network such as an IP internet
in a position normally occupied by physical cabling introduces
some security problems not normally encountered in Fibre Channel
storage networks. Normal media are typically protected physically
from outside access; IP internets typically invite outside
accesses.
The general effect is that the security of the entire Fibre
Channel internetwork is only as good as the security of the
entire IP internet through which it tunnels. The following broad
classes of attacks are possible:
Rajagopal, et al. [Page 12]
Internet-Draft Fibre Channel over TCP/IP October 2000
1. Unauthorized Fibre Channel controllers can gain access to
resources through normal Fibre Channel processes.
2. Unauthorized agents can monitor and manipulate Fibre Channel
traffic flowing over physical media used by the IP internet
and under control of the agent.
To a large extent, these security risks are typical of the risks
facing any other application using an IP internet. They are
mentioned here only because Fibre Channel storage networks are
not normally suspicious of the media. Fibre Channel storage
network administrators will need to be aware of these additional
security risks.
Security protocols and procedures used in other IP applications
may also be used for FC over TCP/IP.
For Virtual Private Networks, both authentication and encryption
are generally desired, because it is important both to (1) assure
that unauthorized users do not penetrate the virtual private net-
network and (2) assure that eavesdroppers on the network cannot
read messages sent over the network.
IPSec provides 3 main facilities: an authentication-only function
referred to as Authentication Header (AH) , a combined
authentication/encryption function called Encapsulating Security
Payload (ESP), and a key exchange function.
Because both features are generally desirable, ESP may be more
suitable than AH. The key exchange function allows for manual
exchange of keys as well as an automated scheme. The IPSec
specifications described in [8], [9], [10], and [11] covers these
topics. It is beyond the scope of this document to discuss
specific uses of the IPSec protocols.
Note: Use of IPSec protocol is optional.
8. Data Integrity Considerations
8.1 Loss
Recovery from data loss due to IP datagram loss is provided via
the TCP reliable delivery mechanism. Note: Due to varying TCP
timeouts, competing FC and TCP recovery schemes is a possibility.
This issue is addressed in section 8.4.
8.2 Corruption
Data corruption is detected at two different levels: TCP checksum
and Fibre Channel CRC. Data corruption detected at the TCP level
shall be recovered via TCP reliable data recovery mechanisms.
Rajagopal, et al. [Page 13]
Internet-Draft Fibre Channel over TCP/IP October 2000
Data corruption detected at the Fibre Channel level shall be
handled within the Fibre Channel end nodes.
8.3 Timeouts
Fibre Channel has two important timeouts to be consider in FCIP.
These are: ED_TOV, and R_A_TOV.
ED_TOV determines the life of an individual Fibre Channel frame
in any particular fabric element. The effects of ED_TOV on the
fabric as a whole are typically cumulative since each fabric
element contains it's own ED_TOV timers for any frame received.
R_A_TOV determines the life of an individual Fibre Channel frame
in the fabric as a whole. For a fabric, R_A_TOV implies that no
particular frame will remain in (and thus be emitted from) the
fabric after the timer expires.
TCP has the TCP acknowledgement timeout. This is a variable
timeout value.
<TBD: Need to elaborate on TCP timeouts and define how Fibre
Channel timeouts map to TCP timeouts.>
8.4 Recovery Mechanisms
When an FC over TCP/IP encapsulated frame is transported over an
IP network, there is a possibility of the frame getting dropped.
This can happen if there is congestion along the path within the
IP network, if there are no empty buffers available on one of the
incoming ports, due to bit errors, etc. When this happens, the
TCP acknowledgement will not be received by the source, and
normal TCP retry mechanisms will be activated.
An issue may arise during these recovery mechanisms, since TCP
timeout is variable, and may exceed Fibre Channel FC ED_TOV/
R_A_TOV timeouts.
<Placeholder: Need to add details on how to handle this here.>
8.5 IP Fragmentation:
The Fibre Channel maximum transmittable unit (MTU) is 2148 bytes.
A maximum of 60 bytes of TCP header increases the MTU of the IP
payload to 2208 bytes. It is preferable that FCIP packets
encompass the TCP + FC MTU to avoid fragmentation of FCIP packets.
The resulting packet size exceeds the MTU of some IP physical
layers (e.g. Ethernet MTU = 1518 bytes). FCIP devices should
handle fragmentation and must handle re-assembly of FCIP packets.
An FCIP device may use Path MTU Discovery (RFC 1191) or an
equivalent mechanism to adjust FCIP packet size to avoid
Rajagopal, et al. [Page 14]
Internet-Draft Fibre Channel over TCP/IP October 2000
fragmentation. Alternatively, the MTUs of all FC nodes may be
manually set to match the path MTU of the IP network.
8.6 TCP Segmentation:
8.7 Ordering:
In an IP network, packets are not guaranteed to be delivered in
the same order in which they were transmitted. But, TCP does
handle ordering. This means that frames will be delivered in the
order that they were transmitted by the FCIP device. Note, this
does not necessarily mean that the frames will be transmitted
by the FCIP device in the order transmitted from the source end
FC device, or received by the destination end FC device in the
order received by the FCIP device. This is due to the fact that
FC frames may be reordered in the FC network/fabric itself.
9. Performance Considerations
Mapping the IP header DSCP bits to correspond to Premium Service
provides a preferred service at each IP Router Per Hop Behavior
(PHB) [6].
The FCIP protocol does not necessitate the "cracking" of the FC
Frame (except for attaching the correct byte-encoded SOF and EOF)
nor does it call for any FC payload processing. This allows any
FC traffic to be tunneled across at high throughput rates.
In the case where there is no data link fragmentation, each FC
frame has a one-to-one mapping to a TCP segment; the TCP segment
has a one-to-one mapping to an IP dategram; and the IP datagram
has a one-to-one mapping to a data link frame. This has the
tendency to further improve the throughput performance.
Note: Class 1 FC traffic expects dedicated bandwidth. This
specification does not address this requirement.
The Flow Control Protocol (discussed in the next section)
provides the ability to stream gigabit FC data when using a large
window size.
10. Flow Control and Congestion Management
<Text to be added>
11. References:
[1] Bradner, S., "The Internet Standards Process -- Revision 3",
BCP 9, RFC 2026, October 1996.
Rajagopal, et al. [Page 15]
Internet-Draft Fibre Channel over TCP/IP October 2000
[2] Bradner, S., "Key words for use in RFCs to Indicate
RequirementLevels", BCP 14, RFC 2119, March 1997
[3] NCITS 321-200x (ANSI) T11/Project 1305-D/Rev 4.7 "Fibre Channel
Switch-Fabric-2", (FC-SW-2) September 29, 2000 (www.t11.org)
[4] Fibre Channel Physical and Signaling Interface-3 (FC-PH-3),
Rev. 9.4, ANSI X3.303-1998
[5] The Fibre Channel Consultant: A Comprehensive Introduction,
"Robert W. Kembel", Northwest Learning Associates, 1998
[6] Nichols, K., Blake, S., Baker, F. and D. Black, " Definition
of the Differentiated Services Field (DS Field) in the IPv4 and
Ipv6 Headers", RFC 2474, December 1998.
[7] NCITS T11/Project 1238-D/Rev4.7 "Fibre Channel Backbone", (FC-
BB) June 8, 2000 (www.t11.org)
[8] Kent, S. and Atkinson, R., "Security Architecture for the
Internet Protocol", RFC 2401, Nov 1998
[9] Kent, S. and Atkinson, R., "IP Authentication Header",
RFC 2402, Nov 1998
[10] Kent, S. and Atkinson, R., "IP Encapsulating Security
Payload (ESP)", RFC 2406, Nov 1998
[11] Maughan, D. et all, "Internet Security Association and Key
Management Protocol (ISAKMP)", RFC 2408, Nov 1998
[12] http://www.isi.edu/in-notes/iana/assignments/protocol-numbers
[13] http://www.t11.org
[14] Fibre Channel Physical and Signaling Interphace (FC-PH), Rev
4.3, ANSI X3.230-1994.
[15] Fibre Channel NCITS 321-200x (ANSI) T11/Project 1356-
D/Rev4.3 " Fibre Channel - Generic Services 3", June 2000
(www.t11.org)).
[16] ISI, "Transmission Control Protocol", RFC 793, Sep 1981
12. Acknowledgments
Rajagopal, et al. [Page 16]
Internet-Draft Fibre Channel over TCP/IP October 2000
13. Authors' Addresses
Murali Rajagopal
LightSand Communications, Inc.
375 Los Coches St.
Milpitas, CA 95035
Phone: +1 408 404 3164
Email: muralir@lightsand.com
Raj Bhagwat
LightSand Communications, Inc.
375 Los Coches St.
Milpitas, CA 95035
Phone: +1 408 404 3194
Fax:
Email: rajb@lightsand.com
Elizabeth G. Rodriguez
Lucent Technologies
1202 Richardson Drive, Suite 210
Richardson, TX 75080
Phone: +1 972 231 0672
Fax: +1 972 671 5476
Email: egrodriguez@lucent.com
Vi Chau
Gadzoox Networks, Inc.
16241 Laguna Canyon Road, Suite 100
Irvine, CA 92618
Phone: +1 949 789 4639
Fax: +1 949 453 1271
Email: vchau@gadzoox.com
Stuart Berman
Vixel Corporation
15245 Alton Parkway Suite 100
Irvine, CA 92618
Phone: +1 949 450 6100
Fax: +1 949 753 9500
Email: sberman@vixel.com
Steve Wilson
Brocade Communications Systems, Inc.
1745 Technology Drive
San Jose, CA. 95110
Rajagopal, et al. [Page 17]
Internet-Draft Fibre Channel over TCP/IP October 2000
Phone: 408-487-8128
Fax: 408-487-8101
email: swilson@brocade.com
Craig W. Carlson
QLogic Corporation
6321 Bury Drive
Eden Prairie, MN 55346
Phone: +1 952 932 4064
Email: craig.carlson@qlogic.com
Michael E. O'Donnell
McDATA Corporation
310 Interlocken Parkway
Broomfield, Co. 80021
Phone: +1 303 460 4142
Fax: +1 303 465 4996
Email: modonnell@mcdata.com
ANNEX A: Fibre Channel EOF and SOF Encodings
A.1 Ordered Sets
On a Fibre Channel link, Ordered Sets (OS) are sent as special
out-of-band words constructed from a combination of the 10-bit
comma character (K28.5) followed by 3 additional 10-bit data
characters. The Ordered Sets defined by Fibre Channel include the
Frame Delimiters, Start of Frame (SOF) and End of Frame (EOF),
other Primitive Signals (IDLE, R_RDY, ARB, OPN, CLS, MRK), and
Primitive Sequences (OLS,NOS, LR, LRR, LIP, LPB, LPE).
When Fibre Chanel frames are encapsulated in an IP packet, the
Byte-encoded frame format is used. The Byte-encoded frame format
uses 32-bit OS Code Words to represent valid Fibre channel frame
delimiters. This format uses a single-byte OS Code to represent
each FC Ordered Set.
FC Over IP makes use of the OS Codes defined in Annex A of [7]
for the frame delimiters. SOF and EOF codes defined in the
figures (see below) in this Annex are inserted into the FC frame.
Primitive Signals and Primitive Sequences are stripped at the
FCIP boundary and not sent between FCIP devices over the IP
network.
Rajagopal, et al. [Page 18]
Internet-Draft Fibre Channel over TCP/IP October 2000
The frame delimiters are identified by their position. An
encapsulated Byte-encoded frame must use the corresponding 32-bit
OS Code Word as the first and last words in the encapsulated PDU.
Fibre Channel frame delimiters shall be encoded in the format
shown in Table below.
Table 1. Frame Delimiter Format
+---+----------------+----------------+----------------+-------------+
|Wrd| <31:24> | <23:16> | <15:08> | <07:00> |
+---+----------------+----------------+----------------+-------------+
|0 | OS Code | Reserved |
+---+----------------+----------------+----------------+-------------+
A.2 Encoded FC Frame Delimiters
The SOF OS-codes are a single byte encoding of the SOF Ordered
Set. The first word in an encapsulated Byte-encoded FC frame
shall map the SOF Ordered Set to the corresponding 32-bit OS Code
Word.
The EOF OS-codes are a single byte encoding of the EOF Ordered
Set. The last word in an encapsulated Byte-encoded FC frame shall
map the the EOF Ordered Set to the corresponding 32-bit OS Code
Word.
+-----------------+----------------+
| OS-Code | Delimiter Name |
| (hex) | |
+-----------------+----------------+
| 0x28 | SOFf |
+-----------------+----------------+
| 0x3F | SOFc1 |
+-----------------+----------------+
| 0x2F | SOFi1 |
+-----------------+----------------+
| 0x37 | SOFn1 |
+-----------------+----------------+
| 0x3D | SOFc2 |
+-----------------+----------------+
| 0x2D | SOFi2 |
+-----------------+----------------+
| 0x35 | SOFn2 |
+-----------------+----------------+
| 0x3E | SOFc3 |
+-----------------+----------------+
| 0x2E | SOFi3 |
+-----------------+----------------+
| 0x36 | SOFn3 |
+-----------------+----------------+
| 0x39 | SOFc4 |
+-----------------+----------------+
| 0x29 | SOFi4 |
Rajagopal, et al. [Page 19]
Internet-Draft Fibre Channel over TCP/IP October 2000
+-----------------+----------------+
| 0x31 | SOFn4 |
+-----------------+----------------+
| 0x38 | SOFcf |
+-----------------+----------------+
| 0x30 | SOFnf |
+-----------------+----------------+
| 0x41 | EOFn |
+-----------------+----------------+
| 0x42 | EOFt |
+-----------------+----------------+
| 0x46 | EOFdt |
+-----------------+----------------+
| 0x44 | EOFrt |
+-----------------+----------------+
| 0x49 | EOFni |
+-----------------+----------------+
| 0x4E | EOFdti |
+-----------------+----------------+
| 0x4F | EOFrti |
+-----------------+----------------+
| 0x50 | EOFa |
+-----------------+----------------+
ANNEX B: Relationship between FC over IP (FCIP, FCoverIP) and IP
over FC (IPFC)
IPFC describes the encapsulation of IP packets in FC frames. It
is intended to facilitate IP communication over an FC network.
FC over IP describes the encapsulation of FC frames in TCP
segments which in turn are encapsulated inside IP packets for
transporting over an IP network. It gives no consideration to the
type of FC frame that is being encapsulated. Therefore, the FC
frame may actually contain an IP packet as described in the IP
over FC specification (RFC 2625). In such a case, the encapsulated
IP packet would have:
IP Header
FC Header
IP Header
TCP header
Note: The two IP headers would not be identical to each other.
One would have information pertaining to the final
destination while the other would have information
pertaining to the FCIP device.
The two documents focus on different objectives. As mentioned
above, implementation of FC over IP will lead to IP encapsulation
Rajagopal, et al. [Page 20]
Internet-Draft Fibre Channel over TCP/IP October 2000
within IP. While perhaps inefficient, this should not lead to
issues with IP communication. One caveat: if a Fibre Channel
device is encapsulating IP packets in an FC frame (e.g. an IPFC
device), and that device is communicating with a device running
IP over a non-FC medium, a second IPFC device will need to act as
a gateway between the two networks. This scenario is not
specifically addressed by FC over IP.
There is nothing in either specification to prevent a single
device from implementing both FC-over-IP and IP-over-FC, but
this is implementation specific, and is beyond the scope of this
document.
Full Copyright Statement
Copyright (C) The Internet Society (1999). All Rights Reserved.
This document and translations of it may be copied and furnished
to others, and derivative works that comment on or otherwise
explain it or assist in its implementation may be prepared,
copied, published and distributed, in whole or in part, without
restriction of any kind, provided that the above copyright notice
and this paragraph are included on all such copies and derivative
works. However, this document itself may not be modified in any
way, such as by removing the copyright notice or references to
the Internet Society or other Internet organizations, except as
needed for the purpose of developing Internet standards in which
case the procedures for copyrights defined in the Internet
Standards process must be followed, or as required to translate
it into languages other than English.
The limited permissions granted above are perpetual and will not
be revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on
an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE
OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY
IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR
PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.
[draft-ietf-ips-fcovertcpip-00.txt] [This INTERNET DRAFT expires
on March 31, 2001]
Rajagopal, et al. [Page 21]
| PAFTECH AB 2003-2026 | 2026-04-19 20:26:00 |