One document matched: draft-ietf-ipfc-fibre-channel-00.txt
IP and ARP Over FC Working Group Murali Rajagopal
INTERNET-DRAFT Raj Bhagwat
<draft-ietf-ipfc-fibre-channel-00.txt> Wayne Rickard
(Expires Dec 22, 1998) (Gadzoox Networks)
IP and ARP over Fibre Channel
Status of this Memo
This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as Reference
material or to cite them other than as ``work in progress''.
To view the entire list of current Internet-Drafts, please check
the "1id-abstracts.txt" listing contained in the Internet-Drafts
Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
(Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au
(Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu
(US West Coast).
Abstract
Fibre Channel (FC) is a high speed serial interface technology that
supports several higher layer protocols including Small Computer
System Interface (SCSI) and Internet Protocol(IP). Until now, SCSI
has been the only widely used protocol over Fibre Channel. Existing
Fibre Channel standards [3] do not adequately specify how IP packets
may be transported over Fibre Channel and how IP addresses are
resolved to FC addresses. The purpose of this document is to specify
a way of encapsulating IP and Address Resolution Protocol(ARP) over
Fibre Channel and also to describe a mechanism for IP address
resolution.
1. Introduction
Fibre Channel is a gigabit speed networking technology primarily used
for Storage Area Networking (SAN). FC is standardized under American
National Standards Institute (ANSI)and has specified a number of
documents describing its protocols, operations, and services.
Need:
Currently, Fibre Channel is predominantly used for communication
between storage devices and servers using the SCSI protocol, with
most of the servers still communicating with each other over LANs.
Although, the Fibre Channel standard [3] has architecturally defined
support for IP encapsulation and address resolution, it is
inadequately specified. ([3] prohibits broadcasts thus loops are not
Rajagopal, Bhagawat, Rickard [Page 1]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
covered; [10] has no support for Class 3)
This has lead to a nonstandard way of using IP over FC in the past.
Once, such a standard method is completely specified, servers can
directly communicate with each other using IP over FC, possibly
boosting performance in Server host-to-host communications. This
technique will be especially useful in a Clustering Application.
Objective:
The major objective of this specification is to promote inter-
operable implementations of IP over Fibre Channel. This
specification describes a method for encapsulating IPv4 and Address
Resolution protocol (ARP) packets over Fibre Channel. This
specification accommodates any FC topology (loop, fabric, or point-
to-point) and any FC class of service (1, 2 or 3). Use of IEEE 802.2
LLC/SNAP encapsulation for IP and ARP as specified in this document
shall not preclude the use of same encapsulation technique for other
protocol stacks (e.g. IPX, AppleTalk).
Organization:
Section 2 states the problem that is solved in this specification.
Section 3 describes the techniques used for encapsulating IP and ARP
packets in a FC sequence.
Section 4 discusses ARP (IP address to MAC address) and the required
mappings and operation. Section 5 discusses the FC Layer mappings
(MAC address to Port_ID). Section 6 provides a discussion on
validation of the FC-layer mapping for the different FC topologies.
Section 7 describes the "Exchange" Management in FC. Section 8 is a
summary section and provides a quick summary of the FC header
settings, FC Link Service Commands, and a summarized reference to
features supported in ARP, FC Sequences, FC Exchanges, and FC Login
Parameters.
Appendix A provides a brief overview of the FC Protocols and Networks
along with a list of acronyms and a glossary of FC Terms used in this
specification. Appendix B addresses reliability in Class 3.
2. Problem Statement
This draft addresses two problems:
- A sequence format definition and encapsulation mechanism for IP
and ARP packets over FC
- An Address Resolution mechanism.
As noted earlier, the existing FC Standards [3], [10] are inadequate.
A solution to both problems has been proposed by the Fibre Channel
Association (FCA)[1]. FCA is a industry consortium of Fibre Channel
vendor companies and not a standards body. This draft specification
is largely based on the proposed solution in [1] and is an attempt to
provide a standardized specification addressing both the above stated
problems.
Rajagopal, Bhagawat, Rickard [Page 2]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
3. IP/ARP Encapsulation
3.1 FC Frame Format
All FC frames have a standard format much like LAN 802.x protocols.
(See Appendix A for Fibre Channel related Acronyms and Glossary of
Terms.) However, the exact size of each frame varies depending on the
sizes of the variable fields. The FC frame structure is shown in
Fig. 1.
+-------+--------+-----------+----//-------+-----+-----+
| SOF |Frame |Optional | Payload |CRC | EOF |
| (4B) |Header |Header | |(4B) |(4B) |
| | |<----------------------->| | |
| |(24B) | (0-2112B) | | |
+-------+--------+-----------+----//-------+-----+-----+
Fig. 1 FC Frame Format
The Start of Frame (SOF) and End of Frame (EOF) are both 4 bytes long
and act as frame delimiters.
The CRC is 4 bytes long and uses the same 32-bit polynomial used in
FDDI and is specified in ANSI X3.139 Fiber Distributed Data
Interface.
The Frame Header is 24 bytes long and has several fields associated
with identification and control of the payload. The values and
options for the fields that are relevant to the IP and ARP payloads
will be discussed later.
A FC Optional Header allows up to 4 optional header fields:
- An Expiration Security Header (16 bytes)
- Network (16 bytes)
- Association (32 bytes)
- Device (up to 64 bytes).
The IP and ARP FC sequences are required to carry the Network_Header
optional header field which is 16 bytes long. Other types of optional
headers are prohibited. The use of the Network_Header for the IP and
ARP payload encapsulation is described below.
In FC an application level payload is called a Sequence. Typically, a
Sequence consists of more than one frame. Larger user data is
segmented and reassembled using two methods: Sequence Count and
Relative Offset. Use of Sequence Count is straight forward and data
blocks are sent using frames with increasing sequence counts (modulo
16). With Relative Offset, frames could temporally arrive out of
order.
When IP and ARP form the FC payload then only the First Frame of the
logical Sequence shall include the FC Network_Header. Care should
exercised when this is the case. Fig. 2 shows the logical First Frame
Rajagopal, Bhagawat, Rickard [Page 3]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
and logical subsequent frames rather than temporal ordering.
First Frame of a Logical FC Sequence
---+------------+---------------------------+----------//----------+---
| FC Header | FC Network Header | FC Sequence Data |
---+------------+---------------------------+---------//-----------+---
Subsequent Frames of a Logical FC Sequence
--+-----------+----------//------+--
| FC Header | FC Sequence Data |
--+-----------+----------//------+--
Fig. 2 FC Network Header in a Frame Sequence
The SOF, CRC, EOF control fields and other optional headers have been
omitted in the figure for clarity.
3.2 MTU
The Maximum Transmission Unit (MTU) for IP is defined as the length
of the IP packet, including IP headers. The theoretical maximum size
of an IP Packet is 65,535 bytes. In FC-4 the transmission unit is
"Information Unit" and not frames. An N_Port may transmit an
Information Unit using multiple frames. The receiving N_Port will
assemble the frames to reconstruct the sent Information Unit. The
size of a single Information Unit is limited to 2^32-1, which is very
large. However, restricting the IP over FC MTU helps in buffer
resource allocation at N_Ports. A MTU of 65,280 bytes allows for up
to 256 bytes of overhead. The IEEE 802.2 LLC/SNAP headers requires 8
bytes, leaving the rest 248 bytes for future uses.
There shall be a one-to-one mapping between an IP packet and a FC
sequence. In other words, one IP packet shall always map to only one
FC Sequence.
Note that, although the FC physical frame MTU is limited to 2112
bytes, it is hidden from IP and does not affect the IP MTU at FC-4.
3.3 FC Port and Node Network Addresses
FC devices are identified by Nodes and Ports. A Node is a collection
of one or more Ports identified by a unique nonvolatile
(unchangeable) 64-bit World Wide Node name (WWN_N). Each Port in a
node, is identified with a unique nonvolatile 64-bit World Wide Port
name (WWP_N), and a volatile (changeable) Port_ID.
Port_ID are 24-bits. In a FC frame header, the Port_ID is referred to
as S_ID (Source ID) to identify the port originating a frame, and
D_ID to identify the destination port. The Port_ID of a given port is
volatile (changeable). (The mechanisms through which a Port_ID may
change in a FC topology are outside the scope of this document.)
FC specifies a Network Address Authority (NAA) to distinguish between
the various name registration authorities that may be used to
Rajagopal, Bhagawat, Rickard [Page 4]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
identify the WWP_N and the WWP_N. A 4-bit NAA identifier, 12-bit
field set to 0x000 and an IEEE 48-bit MAC address together make the
64-bit WWN_N or the WWP_N addresses [2]. In a single port Node, the
WWN_N and the WWP_N may be identical.
The WWN_P names of the source and destinations are carried in the FC
Network Header. The format of the FC Network Header is shown in Fig.
3 and defined in the FC standards [2]. The Network header is
normally optional in FC but mandatory in this specification. The 4
most significant bits in each address denotes the Network Address
Authority (NAA) type. In this specification, the source and
destination NAA binary pattern '0001' indicates the IEEE-48 bit MAC
address and is the only code point that is allowed.
The NAA field allows FC networks to be bridged with other FC networks
or traditional LANs. The Source (Destination) MAC address occupies
the lower 48 bits of the Network_Source_Address
(Network_Dest_Address), and the upper 12 bits are set to 0x000.
+--------+---------------------------------------+
| D_NAA |Network_Dest_Address (High-order bits) |
|(4 bits)| (28 bits) |
+--------+---------------------------------------+
| Network_Dest_Address (Low-order bits) |
| (32 bits) |
+--------+---------------------------------------+
| S_NAA |Network_Source_Address(High-order bits)|
|(4 bits)| (28 bits) |
+--------+---------------------------------------+
| Network_Source_Address (Low-order bit) |
| (32 bits) |
+--------+---------------------------------------+
Fig. 3 Format of the Network Header Field
3.4 FC Payload Format
The payload of an FC sequence carrying an IP packet shall use the
format shown in Fig. 4. Fig. 5 shows the format when the payload is
an ARP packet. However, both formats use the 8-byte LLC/SNAP header.
+-----------------+-----//----------+-------------//------------+
| LLC/SNAP Header | IP Header | IP Data |
| (8 bytes) | (20 bytes min.) | (65280 -IP Header) bytes |
+-----------------+-----//----------+-------------//------------+
Fig. 4 Format of FC Sequence Payload carrying IP
+-----------------+-------------------+
| LLC/SNAP Header | ARP Packet |
| (8 bytes) | (28 bytes) |
+-----------------+-------------------+
Fig. 5 Format of FC Sequence Payload carrying ARP
Rajagopal, Bhagawat, Rickard [Page 5]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
As noted above, since FC frames belonging to the same Sequence can be
delivered out of order over a Fabric, the IP Header must appear in
the frame that has relative random offset of 0.
A Logical Link Control (LLC) field along with a Sub Network Access
Protocol (SNAP) field is a method used to identify routed and bridged
non-OSI protocol PDUs and is defined in IEEE 802.2 and applied to IP
in [8]. In LLC Type 1 operation (i.e., unacknowledged connectionless
mode), the LLC header is 3-bytes long and consists of a 1-byte
Destination Service Access Point (DSAP)field, a 1-byte Source Service
Access Point (SSAP)field, and a 1-byte Control field as shown in Fig.
6.
+----------+----------+----------+
| DSAP | SSAP | CTRL |
| (1 byte) | (1 byte | (1 byte) |
+----------+----------+----------+
Fig. 6 LLC Format
The LLC's DSAP and SSAP values of 0xAA indicate that a SNAP header
follows. The LLC's CTRL value equal to 0x03 specifies Unnumbered
Information Command PDU. The LLC header value shall 0xAA-AA-03.
The SNAP header is 5 bytes long and consists of a 3-byte
Organizationally Unique Identifier (OUI) field and a 2-byte Protocol
Identifier as shown in Fig. 7
+------+------+-------+------+------+
| OUI | PID |
| ( 3 bytes) | (2 bytes) |
+------+------+-------+------+------+
Fig. 7 SNAP Format
The SNAP OUI value 0x00-00-00 specifies that the PID is an EtherType
(i.e., routed non-OSI protocol).
The SNAP PID Type field specifies the EtherType value. In particular,
the value of 0x08-00 indicates IP and value of 0x08-06 indicates ARP.
The complete LLC/SNAP header is shown in Fig. 8.
+----------+----------+----------+-------+-------+-------+-------+------+
| DSAP | SSAP | CTRL | OUI | PID |
| (1 byte) | (1 byte) | (1 byte) | ( 3 bytes) | (2 bytes |
+----------+----------+----------+-------+-------+-------+-------+------+
Fig. 8 LLC/SNAP Header
3.5 ARP Packet Format
The format of the encapsulated ARP packet is based on [9] and is
shown in Fig. 9.
Rajagopal, Bhagawat, Rickard [Page 6]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
The 'HW Type' field shall be set to 0x00-06 indicating IEEE 802
networks.
The 'Protocol' field shall be set to 0x08-00 indicating IP protocol.
The 'HW Addr Length' field shall be set to 0x06 indicating 6 bytes of
HW address.
The 'Protocol Addr Length' field shall be set to 0x04 indicating 4
bytes of IP address.
The 'Operation' Code field shall be either 0x00-01 for Request or
0x00- 02 for Reply.
+-------------------------+
| HW Type | 2 bytes
+-------------------------+
| Protocol | 2 bytes
+-------------------------+
| HW Addr Length | 1 byte
+-------------------------+
| Protocol Addr Length | 1 byte
+-------------------------+
| Op Code | 2 bytes
+-------------------------+
| HW Addr of Sender | 6 bytes
+-------------------------+
| Protocol Addr of Sender | 4 bytes
+-------------------------+
| HW Addr of Target | 6 bytes
+-------------------------+
| Protocol Addr of Target | 4 bytes
+-------------------------+
Fig. 9 ARP Packet Format
The 'HW Addr of Sender' field shall be the 6 byte IEEE MAC address of
the sender.
The 'Protocol Addr of Sender' field shall be the 4 byte IP address of
the sender.
The 'HW Addr of Target' field shall be set to zero if the 'Operation
Code' field is set to 1. Otherwise, it shall be set to the 6 byte
IEEE MAC address of the original sender of the ARP request.
The 'Protocol Addr of Target' field shall be set to the 4 byte IP
address of the target.
The ARP packet is 28 bytes long in this particular application. The
difference between an ARP Request Packet and an ARP Reply Packet is
given below:
1. ARP Request packet: 'Operation' Code field = 0x00-01 and the 'HW
Rajagopal, Bhagawat, Rickard [Page 7]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
Addr of Target' is set to 0x00-00-00-00-00-00.
2. ARP Reply packet: 'Operation' Code field = 0x00-02 and the 'HW
Addr of Target' is appropriately set to the address extracted
from the ARP Request packet's Sender address
An ARP Request message is defined as a FC broadcast sequence carrying
the ARP Request packet. The exact mechanism used to broadcast a FC
sequence depends on the topology and will be discussed in the next
section. Compliant ARP broadcast messages shall include Network
Headers.
An ARP Reply message is defined as an ARP Reply packet encapsulated
in a FC sequence.
4. Address Resolution
4.1 Problem Description
Address Resolution is concerned with associating IP addresses with FC
Port addresses. As described earlier, FC device ports have two
addresses:
- a non-volatile unique 64-bit address called World Wide Port_Name
(WWP_N)
- a volatile 24-bit address called a Port_ID
The Address Resolution mechanism therefore will need two levels of
mapping:
1. A mapping from IP address to the WWP_N address(i.e., IEEE
48-bit MAC address)
2. A mapping from WWP_N to the Port_ID
The address resolution problem is compounded by the fact that the
Port_ID is volatile and the second mapping has to be validated before
use. Moreover, this validation process can be different depending on
the FC network topology used.
Architecturally, the first level of mapping and control operation is
handled by the ARP layer, and the second level of mapping and control
by the FC layer.
4.2 ARP Layer Mapping and Operation
Whenever a source FC port with a designated IP address wishes to send
IP data to a destination FC port also with a designated IP address
then, the following steps are taken:
1. The source port shall consult its local mapping tables to
determine the <destination IP address, destination WWP_N>.
(Note, WWP_N address and 48-bit MAC address will conceptually
mean the same thing in this discussion.)
Rajagopal, Bhagawat, Rickard [Page 8]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
2. If such a mapping is found, then the source shall send the IP
data to the port whose WWP_N address was found in the table. The
corresponding destination Port_ID shall first be validated.
3. If such a mapping is not found, then the source shall send an
ARP broadcast message to its connected FC network in
anticipation of getting a reply from the correct destination
along with its WWP_N address.
4. When an ARP broadcast message is received by the destination it
shall generate an ARP response. Since the ARP response must be
addressed to a specific destination Port_ID, the FC layer
mapping between the MAC address and Port_ID (of the ARP Request
orginator) must be valid before the reply is sent.
4.2.1 ARP Broadcast in a Point-to-Point Topology
There is no requirement for ARP since the WWP_N is known after the
two N_Ports carry out a N_Port Login, that is a PLOGI (See Appendix
A).
4.2.2 ARP Broadcast in a Private Loop Topology
In a private loop, the ARP broadcast message is sent using the
broadcast method specified in the FC-AL [7]standard.
1. The source port shall first send an Open Broadcast
Replicate primitive (OPN(fr))Signal forcing all the ports
in the loop (except itself), to replicate the frames that
they receive while examining the frame header's
Destination_ID field.
2. The source port shall remove this OPN(fr) signal when it
returns to it.
3. The loop is now ready to receive the ARP broadcast message
and is sent as a broadcast sequence, that is using FC
frames.
4. The source shall now send a FC frame containing the ARP
Request (ARP broadcast message), as a sequence in a Class 3
frame with the following FC Header D_ID field and F_CTL bits
in the FC header set to:
Destination ID<Word 0, bit 0:23>: D_ID = 0xFF-FF-FF
Sequence Initiative <Word 2, bit23>: SI=0
Last Sequence <Word 2, bit 20>: LS=1
End Sequence <Word 2, bit 19>: ES=1.
The above FCTL settings apply to single-frame broadcasts,
as used in ARP sequences. This information is provided to
Rajagopal, Bhagawat, Rickard [Page 9]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
clarify ARP Broadcast usage only, and should not be
interpreted as prohibiting the use of multiframe
broadcasts by this specification.
5. Compliant ARP broadcast sequences shall include Network Headers
with destination MAC address in the Network Header set to
0xFF-FF-FF-FF-FF-FF
6. The destination port recognizing its IP address in the ARP
packet shall respond with an ARP Reply message.
4.2.3 ARP Broadcast in a Public Loop Topology
The following steps will be followed when a port is configured in a
public loop:
1. A public loop device attached to a fabric through an FL_Port
shall not use the OPN(fr) signal primitive. Rather, it shall
send the broadcast sequence to the FL_Port at AL_PA = 0x00.
2. A fabric shall propagate the broadcast to all other ports
including the FL_Port which the broadcast arrived on. This
includes all F_Ports, and other FL_Ports.
3. On each FL_Port, the fabric shall first propagate the
broadcast by first using the primitive signal OPNfr, in order
to prepare the loop to receive the broadcast sequence
4. A broadcast sequence is now sent on all ports (all FL_ports,
F_Ports)in Class 3 frame with:
Destination ID <Word 0, bit 23:0>: D_ID = 0xFF-FF-FF
Sequence Initiative <Word 2, bit23>: SI=0
Last Sequence <Word 2, bit 20>: LS=1
End Sequence <Word 2, bit 19>: ES=1.
5. Compliant ARP broadcast sequences shall include Network Headers
with destination MAC address in the Network Header set to
0xFF-FF-FF-FF-FF-FF
6. The destination port recognizing its IP address in the ARP
packet shall respond with an ARP Reply message.
4.2.4 ARP Operation in a Fabric Topology
1. Nodes directly attached to fabric do not require the OPN(fr)
primitive signal.
2. A broadcast sequence is now sent on all ports (all FL_ports,
F_Ports)in Class 3 frame with:
Destination ID <Word 0, bit 23:0>: D_ID = 0xFF-FF-FF
Rajagopal, Bhagawat, Rickard [Page 10]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
Sequence Initiative <Word 2, bit23>: SI=0
Last Sequence <Word 2, bit 20>: LS=1
End Sequence <Word 2, bit 19>: ES=1.
3. Compliant ARP broadcast sequences shall include Network Headers
with destination MAC address in the Network Header set to
0xFF-FF-FF-FF-FF-FF
4. The destination port recognizing its IP address in
the ARP packet shall respond with an ARP Reply
5.0 Mechanisms for Maintaining FC Layer Mappings
FC layer mapping between the MAC address and the Port_ID is
independent of the ARP mechanism and is more closely associated with
the details of the FC protocols. The section presents several
possible mechanisms that may be used for maintaining FC-layer
mappings, that is, to create and maintain MAC Address to Port Address
tables. The preferred method is a configuration and administration
issue, and may be implementation-dependent.
Each method should have some mechanism to ensure PLOGI has completed
successfully before data is sent. A related concern in large networks
is limiting concurrent logins to only those ports with active IP
traffic.
5.1 Login on Cached Mapping Information
This method insulates the level performing LOGIN from the level
interpreting ARP. It is more accommodating of non-ARP mechanisms for
building the FC-layer mapping table.
1. Broadcast messages that carry a Network Header contain the
S_ID
on the FC-header and WWP_N in the Network-header. Caching this
information provides a correlation of Port_ID to WWP_N.
If the received Broadcast message is compliant with this
specification, the WWP_N will be the MAC Address. This method
may also accommodate other NAA types.
2. The WWP_N is "available" if Login has been performed to the
Port_ID and flagged. If login has not been performed, the
WWP_N
is "unavailable".
3. If an outbound packet is destined for a port that is
"unavailable", the cached information is used to look up the
Port_ID.
4. After sending an ELS PLOGI command (Port Login) to the Port,
wait. By waiting for an outbound packet before initiating
login, login resources are reserved only for those ports which
Rajagopal, Bhagawat, Rickard [Page 11]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
wish to establish communication.
5. After Port Login completes (ACC received), the outbound packet
can be forwarded. At this point in time, both ends have the
necessary information to complete their <IP address,
MAC Address, Port_ID> association.
5.2 Login on ARP Parsing
This method performs LOGIN sooner by parsing ARP before passing it up
to higher levels for IP/MAC Address correlation. It requires a low-
level awareness of the IP address, and is therefore protocol-
specific.
1. When an ARP Broadcast Message is received, extract the S_ID
from the FC header and the corresponding
Network_Source_Address from the Network Header.
2. Parse the ARP payload to determine if (a) you are the target
of the ARP request (Target IP Address match), and (b) you are
currently logged in with the port (Port_ID = S_ID) originating
the ARP broadcast.
3. Pass the ARP to higher level for ARP Response generation.
4. If a Port Login is required, an ELS PLOGI command (Port Login)
is sent immediately to the Port originating the ARP Broadcast.
5. After Port Login completes, an ARP response can be forwarded.
Note that there are two possible scenarios:
- The ACC to PLOGI returns before the ARP reply is processed
and the ARP Reply is immediately forwarded.
- The ARP reply is delayed, waiting for ACC (successful
Login).
6. At this point in time, both ends have the necessary
information to complete their
<IP address, MAC Address, Port_ID> association.
5.3 Use of Name Server
This method is preferred in environments where a Name Server is
required [4]. Compliant topologies require a Name Server, while [5]
devices may not be able to access the well-known Name Server address,
even if one exists.
1. A Name Server may be referenced to resolve unmapped MAC
addresses.
2. Any upper layer send request for which there is not a
Port_ID to MAC address mapping can trigger a query to a name
server.
Rajagopal, Bhagawat, Rickard [Page 12]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
3. The format of the Name Server query and response is outside
the scope of this document. See FC-FLA [4] for a typical
example.
4. A preferred Name Server implementation is described in
[ns008.pdf on ftp.network.com]. The MAC address must be
re-formatted in the 64-bit WWP_N format before the query is
issued.
5. The query response from the Name Server must contain the
Port_ID associated with the MAC Address specified in the
query.
6. Send an ELS PLOGI command (Port Login) to the Port.
7. After Port Login completes, the outbound packet can be
forwarded.
8. At this point in time, both ends have the necessary
information to complete their <IP address, MAC Address,
Port_ID association>.
5.4 Login to Everyone
In Fibre Channel topologies with a limited number of ports, it may be
efficient to unconditionally login to each port. This method is
discouraged in fabric and public loop environments.
After Port Login completes, the MAC Address to Port_ID Address tables
can be constructed.
5.5 Static Table
In some loop environments with a limited number of ports, a static
mapping from a MAC Address to Port_ID (D_ID or AL_PA) may be
maintained. The FC layer will always know the destination Port_ID
based on the table. The table is typically downloaded into the driver
at configuration time. This method scales poorly, and is therefore
not recommended.
5.6 FARP
The Fibre Channel Address Resolution Protocol (FARP) is a method
using ELS commands to resolve <WWP_N, D_ID> mapping in environments
without a Name Server. That is, when the WWP_N is known, but not the
D_ID and a Name Server service doesn't exist. This situation arises,
for instance, when Login tables entries expire.
The FARP Extended Link Service Request shall resolve Port_IDs of
communicating Fibre Channel devices. A FARP Request can be used to
retrieve a specific N_Port's current Port_ID given the unique WWP_N
and WWN_N. This is accomplished by requesting either a FARP Response
ELS command, or by indicating that the Responder N_Port shall perform
a login with the FARP Originator.
Rajagopal, Bhagawat, Rickard [Page 13]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
Protocol:
FARP Request Sequence (ELS broadcast)
No Reply Sequence
FARP Response Sequence (ELS command)
Accept Reply Sequence
Format: FT-1
Addressing:
- For a FARP Request, The S_ID designates the Originator
N_Port requesting addressing information. The D_ID is the
broadcast identifier, 0xFF-FF-FF.
- For a FARP Response, the S_ID designates the N_Port ID of the
device matching the Responder Address Information in the FARP
Request. The D_ID is the N_Port ID of the device that initiated
the FARP request.
Payload: The format of the FARP Request payload is as follows:
+-----------------------------------------+---------+
| FARP Request Payload | |
+-----------------------------------------+---------+
| Field | Size |
| |(Bytes) |
+-----------------------------------------+---------+
| 0x54-00-00-00 | 4 |
+-----------------------------------------+---------+
| Responder Flags | 1 |
+-----------------------------------------+---------+
| Port_ID of Originator | 3 |
+-----------------------------------------+---------+
|WWP_N of Originator | 8 |
+-----------------------------------------+---------+
|WWN_N of Originator | 8 |
+-----------------------------------------+---------+
|WWP_N of Responder | 8 |
+-----------------------------------------+---------+
|WWN_N of Responder | 8 |
+-----------------------------------------+---------+
The "WWP_N of Responder" and "WWN_N of Responder" fields should be
filled in with the Node and Port Names of the desired Responder,
while the Responder Flags define what action the Responder should
take. The FARP Request Originator can supply the WWP_N of the
Responder, the WWN_N of the Responder, or both. Corresponding bits
Rajagopal, Bhagawat, Rickard [Page 14]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
in the "Responder Flags" field should also be set.
WWP_N in FARP is the 8-byte WWP_N of the Originator / Responder to
the FARP request.
WWN_N in FARP is the 8-byte WWN_N of the Originator / Responder to
the FARP request.
Port_ID: is the 24-bit Port_ID used in the S_ID field of the FARP
Request or FARP Response header.
Responder Flags: is an 8-bit field (bits 0-7) that defines the action
of the Responder. This field is only valid in a FARP Request.
Table below indicates the action performed for each bit. If no bits
are set, the Responder will take no action.
+----------+-------------------------------------------------------+
| | FARP Responder Flag |
+----------+--------------+----------------------------------------+
| Bit | Bit Name | Action |
| Position | | |
+----------+--------------+----------------------------------------+
| 0 | MATCH_PORT | Match on WWP_N of Responder |
+----------+--------------+----------------------------------------+
| 1 | MATCH_NODE | Match on WWN_N of Responder |
+----------+--------------+----------------------------------------+
| 2 | INIT_PLOGI | Initiate P_LOGI to the Originator |
+----------+--------------+----------------------------------------+
| 3 | INIT_FARPR | Send FARP Response ELS to Originator |
+----------+--------------+----------------------------------------+
| 4 | Reserved | |
+----------+--------------+----------------------------------------+
| 5 | Reserved | |
+----------+--------------+----------------------------------------+
| 6 | Reserved | |
+----------+--------------+----------------------------------------+
| 7 | Reserved | |
+----------+--------------+----------------------------------------+
FARP Request is an ELS broadcast command. You do not have to be
logged in to issue a FARP request.
Possible Responder Actions:
Port Login (P_LOGI)
Sent to the Port Identified by " Originator Port_ID" field
when responder bit 2 (INIT_PLOGI) == binary '1'
FARP Response (FARP) Sequence
Sent to the Port Identified by "Originator Port_ID" field
when responder bit 3 (INIT_FARPR) == binary '1'
Rajagopal, Bhagawat, Rickard [Page 15]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
Recipients of the FARP Request ELS shall not issue a Service Reject
(LS_RJT) if FARP is not supported.
For each recipient of the FARP Request Broadcast ELS, the recipients
WWN_N and/or WWP_N is matched against the "WWN_N of Responder" and
"WWP_N of Responder" fields based on the Responder Flags.
If the MATCH_PORT bit is set, the Responder WWP_N is compared with
the recipients WWP_N.
If the MATCH_NODE bit is set, the Responder WWN_N is compared with
the recipients WWN_N.
If both bits are set, both are compared, and both have to match. If
no match is made, the sequence is ignored and no action is taken. If
there is a match, the "Responder Flags" field defines what action to
take. This logic is shown in the following table:
+-------------------+-------------------+-------------------+
| Compare | MATCH_PORT | MATCH_NODE |
+-------------------+-------------------+-------------------+
| Ignore | 0 | 0 |
+-------------------+-------------------+-------------------+
| Compare | | |
| Responder WWP_N | 1 | 0 |
| with | | |
| Recipient WWP_N | | |
+-------------------+-------------------+-------------------+
| Compare | | |
| Responder WWN_N | 0 | 1 |
| with | | |
| Recipient WWN_N | | |
+-------------------+-------------------+-------------------+
| Compare | | |
| Responder WWP_N & | | |
| WWN_N | 1 | 1 |
| with | | |
| Recipient WWP_N | | |
| & WWN_N | | |
+-------------------+-------------------+-------------------+
FARP Response is an ELS command directed to the Originator of the
FARP Request. You do not have to be logged in to the FARP Request
Originator to issue a FARP Response.
Reply Link Service Sequence:
Service Reject (LS_RJT)
Signifies rejection of the FARP Response command
Accept (ACC) Reply Sequence
Signifies successful completion of the FARP Response
command
Rajagopal, Bhagawat, Rickard [Page 16]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
The format of the FARP Response payload is as follows:
+-----------------------------------------+---------+
| FARP Response Payload | |
+-----------------------------------------+---------+
| Field | Size |
| | (Bytes) |
+-----------------------------------------+---------+
| 0x54-01-00-00 | 4 |
+-----------------------------------------+---------+
| Reserved | 1 |
+-----------------------------------------+---------+
| Port_ID of Responder | 3 |
+-----------------------------------------+---------+
|WWP_N of Originator (FARP Request) | 8 |
+-----------------------------------------+---------+
|WWN_N of Originator (FARP Request) | 8 |
+-----------------------------------------+---------+
|WWP_N of Responder | 8 |
+-----------------------------------------+---------+
|WWN_N of Responder | 8 |
+-----------------------------------------+---------+
Accept Payload:
The format of the FARP Accept payload is as follows:
+-----------------------------------------+---------+
| FARP Response Accept Payload | |
+-----------------------------------------+---------+
| Field | Size |
| |(Bytes) |
+-----------------------------------------+---------+
| x02-00-00-00 | 4 |
+-----------------------------------------+---------+
6.0 FC layer Address Validation
At all times, the <MAC Address, Port_ID> mapping has to be validated
before use. There are many events that can invalidate this mapping.
The following discussion addresses conditions when such a validation
is required.
6.1 General Discussion
After a link interruption occurs, the Port_ID of a port may change.
After the interruption, the Port_IDs of all other ports that have
previously performed PLOGI (N_Port Login) with this port may have
changed, and its own Port_ID may have changed.
Because of this, address validation is required after a LIP in a loop
topology [7]or after NOS/OLS in a point-to-point topology [6].
Rajagopal, Bhagawat, Rickard [Page 17]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
Port_IDs will not change as a result of Link Reset(LR),thus address
validation is not required.
In addition to actively validating devices after a link interruption,
if a port receives any FC-4 data frames (other than broadcast
frames), from a port that is not currently logged in, then it shall
send an explicit Extended Link Service (ELS) Request logout (LOGO)
command to that port.
ELS commands (Requests and Replies) are used by an N_Port to solicit
a destination port (F_Port or N_Port) to perform some link-level
function or service.) The LOGO Request is used to request
invalidation of the service parameters and Port_ID of the recipient
N_Port.
The level of initialization and subsequent validation and recovery
reported to the upper (FC-4) layers is implementation-specific.
In general, an explicit Logout (LOGO) shall be sent whenever the FC-
Layer mapping between the Port_ID and WWP_N of a remote port is
removed.
The effect of power-up or re-boot on the mapping tables is outside
the scope of this specification.
6.2 FC Layer Address Validation in a Point-to-Point Topology
No validation is required after LR. In a point-to-point topology,
NOS/OLS causes implicit logout of each port and after a NOS/OLS, each
port must perform a PLOGI [2].
6.3 FC Layer Address Validation in a Private Loop Topology
After a LIP, a port shall not transmit any link data to another port
until the address of the other port has been validated. The
validation consists of completing either ADISC or PDISC. (See
Appendix A)
ADISC (Address Discovery) is an ELS command for discovering the hard
addresses - the 24-bit NL_port identifier- of N_Ports [5], [6].
PDISC (Discover Port) is an ELS command for exchanging service
parameters without affecting login state [5], [6].
As a requester, this specification prohibits PDISC and requires
ADISC.
As a responder, an implementation may need to respond to both ADISC
and PDISC for compatibility with other FC specifications.
If the three addresses, Port_ID, WWP_N, WWN_N, exactly match the
values prior to the LIP, then any active exchanges may continue.
If any of the three addresses have changed, then the node must be
Rajagopal, Bhagawat, Rickard [Page 18]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
either implicitly or explicitly logged out [4], [5].
6.4 FC Layer Address Validation in a Public Loop Topology
After a LIP, each public loop port shall not transmit any frame until
it receives the FAN ELS from the fabric [4].
A FAN (Fabric Address Notification) ELS command is sent by the fabric
to all known previously logged in ports following an initialization
event.
The WWP_N and WWN_N of the fabric FL_Port contained in the FAN ELS
must exactly match the values before the LIP. In addition, the AL_PA
obtained by the port must be the same as the one before the LIP.
If the above conditions are met, the port may resume all exchanges.
If not, then FLOGI (Fabric login) must be performed with the fabric
and all nodes must be either implicitly or explicitly logged out.
A public loop device will have to perform the private loop
authentication to any nodes on the local loop which have an Area +
Domain Address == 0x00-00-XX
6.5 FC Layer Address Validation in a Fabric Topology
No validation is required after LR (link reset).
After NOS/OLS, a port must perform FLOGI. If, after FLOGI, the S_ID
of the port, the WW Port Name of the fabric, and the WWN_N of the
fabric are the same as before the NOS/OLS, then the port may resume
all exchanges. If not, all nodes must be either, implicitly or
explicitly, logged out [2].
7. Exchange Management
7.1 Exchange Origination
FC Exchanges shall be established to transfer data between ports.
Frames on IP exchanges shall not transfer Sequence Initiative.
7.2 Exchange Termination
With the exception of the recommendations in Appendix C, "Reliability
in Class 3", the mechanism for aging or expiring exchanges based on
activity, timeout, or other method is outside the scope of this
document.
Exchanges may be terminated by either port.
The Exchange Originator shall normally terminate Exchanges by setting
the LS bit, following normal FC standard FC-PH [2] rules. This
specification prohibits the use of the NOP ELS with LS set for
Exchange termination.
Rajagopal, Bhagawat, Rickard [Page 19]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
Exchanges may be torn down by the Exchange Responder by using the
ABTS_LS protocol. The use of ABTS_LS for terminating aged exchanges
or error recovery is outside the scope of this document.
The termination of IP exchanges by Logout is discouraged, since this
may terminate active exchanges on other FC-4s.
8. Summary of Supported Features
Note: 'Required' means the feature support is mandatory, 'Prohibited'
means the feature support is not allowed, 'Allowed' means the feature
support is optional, and 'Settable' means support is as specified in
the relevant standard.
8.1 FC-4 Header (Note 1)
+--------------------------------------------------------------------+
| Feature | Support | Notes |
+--------------------------------------------------------------------+
| Type Code ( = 5) ISO8802-2 LLC/S | Required | 2 |
| Network Headers | Required | 3 |
| Other Optional Headers | Prohibited | |
+--------------------------------------------------------------------+
Notes:
1. This table applies only to FC-4 related data, such as IP and
ARP packets. This table does not apply to link services and
other non-FC-4 sequences (PLOGI, for example) that must occur
for normal operation.
2. The TYPE field in the FC Header (Word 2 bits 31-24) must
indicate ISO 8802-2 LLC/SNAP Encapsulation (Type 5). This
revision of the document focuses solely on the issues related
to running IP and ARP over FC. All other issues are outside
the scope of this document, including full support for IEEE
802.2 LLC.
3. DF_CTL field (Word 3, bits 23-16 of FC-Header)must indicate
the presence of a Network Header (0010 0000) on the First
logical Frame of FC-4 sequences.
8.2 R_CTL (FC-Header Word 0, bits 31-14)
+--------------------------------------------------------------------+
| Feature | Support | Notes |
+--------------------------------------------------------------------+
| Information Category (R_CTL Routing): | | |
| FC-4 Device Data | Required | 1 |
| Extended Link Data | Required | 2 |
| FC-4 Link Data | Prohibited | |
| Video Data | Prohibited | |
| Basic Link Data | Required | 3 |
| Link Control | Required | 4 |
| R_CTL information | | |
| Uncategorized | Prohibited | |
Rajagopal, Bhagawat, Rickard [Page 20]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
| Solicited Data | Prohibited | |
| Unsolicited Control | Required | 2 |
| Solicited Control | Required | 2 |
| Unsolicited Data | Required | 1 |
| Data Descriptor | Prohibited | |
| Unsolicited Command | Prohibited | |
| Command Status | Prohibited | |
+--------------------------------------------------------------------+
Notes:
1. This is required for FC-4 (IP and ARP) packets
- Routing bits of R_CTL field must indicate Device Data
frames (0000).
- Information Category of R_CTL field must indicate
Unsolicited Data (0100).
2. This is required for Extended Link Services.
3. This is required for Basic Link Services.
4. This is required for Link Control frames.
8.3 F_CTL (FC-Header Word 2, bits 23-0)
+--------------------------------------------------------------------+
| Feature | Support | Notes |
+--------------------------------------------------------------------+
| Exchange Context | Settable | |
| Sequence Context | Settable | |
| First / Last / End Sequence (FS/LS/ES) | Settable | |
| Chained Sequence | Prohibited | |
| Sequence Initiative (SI) | Settable | 1 |
| X_ID Reassigned / Invalidate | Prohibited | |
| Unidirectional Transmit | Settable | |
| Continue Sequence Condition | Required | 2 |
| Abort Seq. Condition -continue and single seq.| Required | 3 |
| Relative Offset - Unsolicited Data | Settable | 4 |
| Fill Bytes | Settable | |
+--------------------------------------------------------------------+
Notes:
1. For FC-4 frames, each N_Port shall have a dedicated X_ID for
sending data to each N_Port in the network and a dedicated
X_ID for receiving data from each N_Port as well. Exchanges
are used in a unidirectional mode, thus setting sequence
initiative is not valid for FC-4 frames. Sequence initiative
is valid when using Extended Link Services.
2. This field is required to be 00, no information.
3. Sequence error policy is requested by an exchange originator
in the F_CTL Abort Sequence Condition bits in the first data
frame of the exchange. For classes 1 and 2, ACK frame is
required to be "continuous sequence".
4. Relative offset prohibited on all other types (Information
Rajagopal, Bhagawat, Rickard [Page 21]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
Category) of frames.
8.4 Sequences
+---------------------------------------------------------------------+
| Feature | Support |Notes |
+---------------------------------------------------------------------+
| Class 2 open sequences / exchange | 1 | 1 |
| Length of seq. not limited by end-to-end credit | Required | 2 |
| Maximum sequence size - IP sequences | 65536 | 3 |
| Maximum sequence size - ARP sequences | 532 | 4 |
| Capability to receive sequence of maximum size | Allowed | 5 |
| Sequence Streaming | Prohibited | 6 |
| Stop Sequence Protocol | Prohibited | |
| ACK_0 support | Allowed | 7 |
| ACK_1 support | Required | 7 |
| ACK_N support | Prohibited | |
| Class of Service for transmitted sequences | 1, 2 or 3 | 8 |
| Continuously Increasing Sequence Count | Allowed | 9,10 |
+---------------------------------------------------------------------+
Notes:
1. Only one active sequence per exchange is allowed.
2. A sequence initiator shall be capable of transmitting
sequences containing more frames than the available credit
indicated by a sequence recipient at login. FC-PH [2] end-to
end flow control rules will be followed when transmitting such
sequences.
3. Maximum sequence size is 65536 bytes. Thus the maximum IP
packet size (MTU) is 65280 bytes (65536 - 256 bytes for header
overhead).
4. Maximum size ARP packet is 532 bytes (including LLC/SNAP
headers).
5. Some OS environments may not handle the max MTU of 65536. It
is up to the administrator to configure the Max MTU for all
systems.
6. All class 3 sequences are assumed to be non-streamed.
7. Only applies for Class 1 and 2. Use of ACK_1 is default,
ACK_0 used if indicated by sequence recipient at login.
8. The administrator configured class of service is used, except
where otherwise specified (e.g. Broadcasts are always sent in
class 3).
9. Review Appendix C, "Reliability in Class 3".
10. The first frame of the first sequence of anew exchange must
have SEQ_CNT = 0 [2].
Rajagopal, Bhagawat, Rickard [Page 22]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
8.5 Exchanges
+--------------------------------------------------------------------+
| Feature | Support | Notes |
+--------------------------------------------------------------------+
| X_ID interlock support | Allowed | 1 |
| OX_ID=FFFF | Prohibited | |
| RX_ID=FFFF | Allowed | 2 |
| Action if no exchange resources available | P_RJT | 3 |
| Long Lived Exchanges | Allowed | 4 |
| Reallocation of Idle Exchanges | Allowed | |
+--------------------------------------------------------------------+
Notes:
1. Only applies to Classes 1 and 2, supported by the exchange
originator. A Port shall be capable of interoperating with
another Port that requires X_ID interlock. The exchange
originator facility within the Port shall use the X_ID
Interlock protocol in such cases.
2. An exchange responder is not required to assign RX_IDs. If a
RX_ID of FFFF is assigned, it is identifying exchanges based
on S_ID / D_ID / OX_ID only.
3. In Classes 1 and 2, a Port shall reject a frame that would
create a new exchange with a P_RJT containing reason code
"Unable to establish exchange". In Class 3, the frame would be
dropped.
4. When an exchange is created between 2 Ports for IP/ARP data,
it remains active while the ports are logged in with each
other. An exchange shall not transfer Sequence Initiative
(SI). Broadcasts and ELS commands may use short lived
exchanges.
8.6 ARP
+--------------------------------------------------------------------+
| Feature | Support | Notes |
+--------------------------------------------------------------------+
| ARP Server Support | Prohibited | 1 |
| Response to ARP requests | Required | 2 |
| ARP requests transmitted as broadcast message | Required | |
| Class of Service for ARP requests | 3 | 3 |
| Class of Service for ARP replies | 1, 2 or 3 | 4 |
+--------------------------------------------------------------------+
Notes:
1. Well-known Address FFFFFC is not used for ARP requests. Frames
from Well-known Address FFFFFC are not considered to be ARP
frames. Broadcast support is required for ARP.
2. The IP Address is mapped to a specific MAC address with ARP.
Rajagopal, Bhagawat, Rickard [Page 23]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
3. An ARP request is a broadcast message, thus Class 3 is always
used.
4. An ARP reply is a normal sequence, thus the administrator
configured class of service is used.
8.7 Extended Link Services (ELS)
+--------------------------------------------------------------------+
| Feature | Support | Notes |
+--------------------------------------------------------------------+
| Class of service for ELS commands / responses | 1,2 or 3 | 1 |
| Explicit N-Port Login | Required | |
| Explicit F-Port Login | Required | |
| FLOGI ELS command | Required | |
| PLOGI ELS command | Required | |
| ADISC ELS command | Required | |
| PDISC ELS command | Allowed | 2 |
| FAN ELS command | Required | 3 |
| LOGO ELS command | Required | |
| Other ELS command support | Allowed | 4 |
+--------------------------------------------------------------------+
Notes:
1. The administrator configured class of service is used.
2. PDISC is prohibited as requester. ADISC should be used
instead. As a responder, an implementation may need to respond
to both ADISC and PDISC for compatibility with other
specifications.
3. FAN is required in a public loop environment.
4. If other ELS commands are received an LS_RJT may be sent. NOP
is not required by this specification, and should not be used
as a mechanism to terminate exchanges.
8.8 Login Parameters
Unless explicitly noted here, a compliant implementation shall use
the login parameters as described in [4].
8.8.1 Common Service Parameters - FLOGI
- FC-PH Version, lowest version may be 0x09 to indicate
'minimum 4.3'.
- Can't use BB_Credit=0 for N_Port on a switched Fabric
(F_Port).
8.8.2 Common Service Parameters - PLOGI
- FC-PH Version, lowest version may be 0x09 to indicate
'minimum 4.3'.
Rajagopal, Bhagawat, Rickard [Page 24]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
- Can't use BB_Credit=0 for N_Port in a Point-to-Point
configuration
- Random Relative Offset is allowed.
- Note that the 'Receive Data Field Size' fields specified in
the PLOGI represent both optional headers and payload.
- The MAC Address can therefore be extracted from the 6 lower
bytes of the WWP_N field (when the IEEE 48-bit Identifier
format is chosen as the NAA) during PLOGI or ACC payload
exchanged during Fibre Channel Login [2].
- The MAC Address can also be extracted from the WWP_N field in
the Network Header during ADISC (and ADISC ACC), or PDISC
(and PDISC ACC).
8.8.3 Class 3 Service Parameters - PLOGI
- Discard error policy only.
ACKNOWLEDGEMENT
This specification is based on FCA IP Profile, Version 3.3. The FCA
IP Profile was a joint work of the Fibre Channel Association (FCA)
vendor community. The following companies and organizations have
contributed to the creation of the FCA IP Profile: Adaptec, Ancor,
Brocade, Clarion, Crossroads, emf Associates, Emulex, Finisar,
Gadzoox, Hewlett Packard, Interphase, Jaycor, LLNL, McData, Migration
Associates, Prisa, Q-Logic, Symbios, Systran, Tektronix, Univ. of
Minnesota, Univ. of New Hamshire.
REFERENCES
[1] FCA IP Profile, Revision 2.3, May 15, 1997
[2] Fibre Channel Physical and Signaling Interface (FC-PH) , ANSI
X3.230-1994
[3] Fibre Channel Link Encapsulation (FC-LE), Revision 1.1, June 26,
1996
[4] Fibre Channel Fabric Loop Attachment (FC-FLA), Rev. 2.4, October
21, 1996
[5] Fibre Channel Private Loop SCSI Direct Attach (FC-PLDA),
Rev.1.7, October 7, 1996
[6] Fibre Channel Physical and Signaling Interface-2 (FC-PH-2),
Rev. 7.4, ANSI X3.297-1996
[7] Fibre Channel Arbitrated Loop (FC-AL), ANSI X3.272-1996
[8] Postel, J. and Reynolds, J., "A standard for the Transmission of
Rajagopal, Bhagawat, Rickard [Page 25]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
IP Datagrams over IEEE 802 Networks". RFC 1042, ISI, Feb, 1988
[9] Plummer, D. "An Ethernet Address Resolution Protocol -or-
Converting Network Addresses to 48-bit Ethernet Address for
Transmission on Ethernet Hardware", STD 37, RFC 826, MIT, Nov
1982.
[10] FCSI IP Profile, FCSI-202, Revision 2.1, September 8, 1995
[11] Fibre Channel Physical and Signaling Interface -3 (FC-PH-3),
Rev. 9.1, ANSI X3.xxx-199x
[12] Fibre Channel-The Basics, "Gary R. Stephens and Jan V. Dedek",
Ancot Corporation
[13] Fibre Channel -Gigabit Communications and I/O for Computers
Networks "Alan Benner", McGraw-Hill, 1996, ISBN 0-07-005669-2
AUTHORS' ADDRESSES
Murali Rajagopal
Gadzoox Networks, Inc.
711 Kimberly Avenue, Suite 100
Placentia, CA 92870
Phone: +1 714 577 6805
Fax: +1 714 524 8508
Email: murali@gadzoox.com
Raj Bhagwat
Gadzoox Networks, Inc.
711 Kimberly Avenue, Suite 100
Placentia, CA 92870
Phone: +1 714 577 6806
Fax: +1 714 524 8508
Email: raj@gadzoox.com
Wayne Rickard
Gadzoox Networks, Inc.
711 Kimberly Avenue, Suite 100
Placentia, CA 92870
Phone: +1 714 577 6803
Fax: +1 714 524 8508
Email: wayne@gadzoox.com
APPENDIX - A
FIBRE CHANNEL OVERVIEW
A.1 Brief Tutorial
FC standard [2] defines 4 "levels" (not layers) for its protocol description: FC-0,
FC-1, FC-2, FC-3, and FC-4. The first three levels (FC-0, FC-1, FC-2)
Rajagopal, Bhagawat, Rickard [Page 26]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
are largely concerned with the physical formatting and control aspects
of the protocol. FC-3 is architecturally defined but not unspecified at this time.
FC-4 is meant for supporting profiles of higher protocols such as IP and
Small Computer Serial Interface (SCSI) and supports a relatively small
set of higher level protocols compared to LAN protocols such as IEEE
802.3.
FC devices are called "Nodes", each of which has at least one "Port" to
connect to other ports. A Node may be a workstation, a disk drive or
disk array, a camera, a display unit, etc. The set of hardware components,
and transreceivers, connecting two or more node ports is called a topology.
A "Link" is two unidirectional paths flowing in opposite directions and
connecting two Ports within adjacent Nodes.
FC Nodes communicate using these higher layer protocols such as SCSI and IP
over FC and are configured to operate using one of the following
networking topologies:
- Point-to-Point
- Private Loop
- Public Loop (attachment to a Fabric)
- Fabric
The point-to-point is the simplest of the four topologies, where only
two nodes communicate with each other. The private loop may connect a
number of devices (max 126) in a logical ring much like Token Ring and
is distinguished from a public loop by the absence of a Fabric Node
participating in the loop. The Fabric topology is a switched network
where any attached node can communicate with any other.
Table below summarizes the usage of port types depending on its location
[12]:
+-----------+-------------+-----------------------------------------+
| Port Type | Location | Topology Associated with |
+-----------+-------------+-----------------------------------------+
| N_Port | Node | Point-to-Point or Fabric |
+-----------+-------------+-----------------------------------------+
| NL_Port | Node |In N_Port mode -Point-to-Point or Fabric |
| | |In NL_Port mode - Arbitrated Loop |
+-----------+-------------+-----------------------------------------+
| F_Port | Fabric | Fabric |
+-----------+-------------+-----------------------------------------+
| FL_Port | Fabric | In F_Port mode - Fabric |
| | | In FL_Port mode - Arbitrated Loop |
+-----------+-------------+-----------------------------------------+
| E_Port | Fabric | Internal Fabric Expansion |
+-----------+-------------+-----------------------------------------+
| G_Port | Fabric | In F_Port mode - Fabric |
| | | In E_Port mode - Internal Fabric Expan.|
+-----------+-------------+-----------------------------------------+
| GL_Port | Fabric | In F_Port mode - Fabric |
| | | In FL_Port mode - Abritrated Loop |
| | | In E_Port mode - Internal Fabric Expan. |
+-----------+-------------+-----------------------------------------+
Rajagopal, Bhagawat, Rickard [Page 27]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
A.2 Fibre Channel Header Fields
Fibre Channel Frame Header, Network Header, and payload carrying IP Packet
+---+----------------+----------------+----------------+--------------+
|Wrd| <31:24> | <23:16> | <15:08> | <07:00> |
+---+----------------+----------------+----------------+--------------+
|0 | RTCL | D_ID |
+---+----------------+----------------+----------------+--------------+
|1 | RSVD | S_ID |
+---+----------------+----------------+----------------+--------------+
|2 | TYPE | F_CTL |
+---+----------------+----------------+----------------+--------------+
|3 | SEQ_ID | DF_CTL | SEQ_CNT |
+---+----------------+----------------+----------------+--------------+
|4 | OX_ID | RX_ID |
+---+----------------+----------------+----------------+--------------+
|5 | NAA | Network_Dest_Address (MSB) |
+---+----------------+----------------+----------------+--------------+
|6 | Network_Dest_Address (LSB) |
+---+----------------+----------------+----------------+--------------+
|7 | NAA | Network_Src_Address (MSB) |
+---+----------------+----------------+----------------+--------------+
|8 | Network_Src_Address (LSB) |
+---+----------------+----------------+----------------+--------------+
|9 | DSAP | SSAP | CTRL | OUI |
+---+----------------+----------------+----------------+--------------+
|10 | OUI | PID |
+---+----------------+----------------+----------------+--------------+
|11 | IP Packet Data |
+---+----------------+----------------+----------------+--------------+
|12 | ... |
+---+----------------+----------------+----------------+--------------+
The FC header as shown in the above diagrams contains routing and other
control information to manage frames, sequences, and exchanges. The
frame header is sent as 6 transmission words immediately following an SOF
delimiter and before the data field.
D_ID and S_ID:
FC uses destination address routing [12], [13]. Frame routing in
a point-to-point topology is trivial.
For the Arbitrated Loop topology, with the destination NL_Port on
the same AL, the source port must pick the destination port,
determine its AL Physical Address, and "Open" the destination
port. The frames must pass through other NL_Ports or the FL_Port
on the loop between the source and destination, but these ports
do not capture the frames. They simply repeat and transmit the
frame. Either communicating port may "Close" the circuit.
When the destination port is not on the same AL, the source
NL_Port must open the FL_Port attached to a Fabric. Once in the
Rajagopal, Bhagawat, Rickard [Page 28]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
Fabric, the Fabric routes the frames again to the destination.
In a Fabric topology, the Fabric looks into the frame header,
extracts the destination address (D_ID), searches its own routing
tables, and sends the frame to the destination port along the path
chosen. The process of choosing a path may be performed at each
fabric until the F_Port attached to the destination N_Port is
reached.
R_CTL (Routing Control) and TYPE(data structure):
Frames for each FC-4 can be easily distinguished from the others
at the receiving port using the R_CTL (Routing Control) and TYPE
(data structure) fields in the frame header.
The R_CTL has two sub-fields: Routing bits and Information category.
The Routing bits sub-field has specific values that mean FC-4 data
follows and the Information Category tells the receiver the "Type" of
data contained in the frame. The R_CTL and TYPE code points are
shown in the diagrams.
Other Header fields:
F_CTL (Frame Control) and SEQ_ID (Sequence Identification),
SEQ_CNT (Sequence Count), OX_ID (Originator exchange Identifier),
RX_ID (Responder exchange Identifier), and Parameter fields are
used to manage the contents of a frame, and mark information
exchange boundaries for the destination port.
F_CTL(Frame Control):
The FC_CTL field is a 3-byte field that contains information
relating to the frame content. Most of the other frame header
fields are used for frame identification. Among other things,
bits in this field indicate the first sequence, last sequence, or
end sequence. Sequence Initiative bit is used to pass control of
the next sequence in the exchange to the recipient.
SEQ_ID (Sequence Identifier) and SEQ_CNT (Sequence Count):
This is used to uniquely identify sequences within an Exchange.
The <S_ID, D_ID, SEQ_ID> uniquely identifies any active sequence.
SEQ_CNT is used to uniquely identify frames within a Sequence to
assure sequentiality of frame reception, and to allow unique
correlation of link control frames with their related data frames.
Originator Exchange Identifier (OX_ID) and Responder Exchange
Identifier (RX_ID):
The OX_ID value provides association of frames with specific
Exchanges originating at a particular N_Port. The RX_ID field
provides the same function that the OX_ID provides for the
Exchange Originator. The OX_ID is meaningful on the Exchange
Originator, and the RX_ID is meaningful on the Responder.
Rajagopal, Bhagawat, Rickard [Page 29]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
DF_CTL (Data Field Control):
The DF_CTL field specifies the presence or absence of optional
headers between the Frame header and Frame Payload
PARAMETER:
The Parameter field has two meanings, depending on Frame type.
For Link Control Frames, the Parameter field indicates the
specific type of link Link Control frame. For Data frames, this
field contains the Relative Offset value. This specifies an
offset from an Upper Layer Protocol buffer from a base address.
Code Points for FC Frame with IP/ARP packet Data
+---+----------------+----------------+----------------+--------------+
|Wrd| <31:24> | <23:16> | <15:08> | <07:00> |
+---+----------------+----------------+----------------+--------------+
| 1 | 0x04 | D_ID |
+---+----------------+----------------+----------------+--------------+
| 2 | 0x00 | S_ID |
+---+----------------+----------------+----------------+--------------+
| 3 | 0x05 | F_CTL |
+---+----------------+----------------+----------------+--------------+
| 4 | SEQ_ID | 0x20 | SEQ_CNT |
+---+----------------+----------------+----------------+--------------+
| 5 | OX_ID | RX_ID |
+---+----------------+----------------+----------------+--------------+
| 6 | 0001 | 0x00-00-00 Dest. MAC |
+---+----------------+----------------+----------------+--------------+
| 7 | Dest. MAC (LSB) |
+---+----------------+----------------+-------------------------------+
| 8 | 0001 | 0x00-00-00 Src. MAC |
+---+----------------+----------------+----------------+--------------+
| 9 | Src. MAC (LSB) |
+---+----------------+----------------+----------------+--------------+
|10 | 0xAA | 0x00 | 0x03 | 0x00 |
+---+----------------+----------------+----------------+--------------+
|11 | 0x00-00 | 0x08-00 |
+---+----------------+----------------+----------------+--------------+
|12 | IP/ARP Packet Data |
+---+----------------+----------------+----------------+--------------+
|13 | ... |
+---+----------------+----------------+----------------+--------------+
A.3 Acronyms and Glossary of FC Terms
It is assumed that the reader is familiar with the terms and acronyms
used in the FC protocol specification [2]. The following is provided for
easy reference.
A.3.1 Acronyms
First Frame: The frame that contains the SOFi field. This means a logical first and may
Rajagopal, Bhagawat, Rickard [Page 30]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
not necessarily be the first frame temporally received in a sequence.
Code Point: The coded bit pattern associated with control fields in frames or packets.
PDU: Protocol Data Unit
ABTS_LS: Abort Sequence Protocol - Last Sequence. A protocol for
aborting an exchange based on the ABTS recipient setting the
Last_Sequence bit in the BA_ACC ELS to the ABTS
ADISC: Discover Address. An ELS for discovering the Hard Addresses (the
24 bit NL_Port Identifier) of N_Ports
D_ID: Destination ID
ES: End sequence. This FCTL bit in the FC header indicates this frame is
the last frame of the sequence.
FAN: Fabric Address Notification. An ELS sent by the fabric to all known
previously logged in ports following an initialization event.
LIP: Loop Initialization. A primitive sequence used by a port to detect
if it is part of a loop or to recover from certain loop errors.
LR: Link reset. A primitive sequence transmitted by a port to initiate
the link reset protocol or to recover from a link timeout.
LS: Last sequence of Exchange. This FCTL bit in the FC header indicates
the sequence is the last sequence of the exchange.
NOS: Not Operational. A primitive sequence transmitted to indicate that
the port transmitting this sequence has detected a link failure or is
offline, waiting for OLS to be received.
OLS: Off line. A primitive sequence transmitted to indicate that the
port transmitting this sequence is either initiating the link
initialization protocol, receiving and recognizing NOS, or entering the
offline state.
PDISC: Discover Port. An ELS for exchanging Service Parameters without
affecting login state.
SI: Sequence Initiative
FLOGI: Fabric Login.
Primitive Sequence: A primitive sequence is an Ordered Set that is
transmitted repeatedly and continuously.
Private Loop Device: A device that does not attempt fabric login (FLOGI)
and usually adheres to PLDA. The Area and Domain components of the
NL_Port ID must be 0x0000. These devices cannot communicate with any
port not in the local loop.
Rajagopal, Bhagawat, Rickard [Page 31]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
Public Loop Device: A device whose Area and Domain components of the
NL_Port ID cannot be 0x0000. Additionally, to be FLA compliant, the
device must attempt to open AL_PA 0x00 and attempt FLOGI. These devices
communicate with devices on the local loop as well as devices on the
other side of a Fabric.
Link: Two unidirectional paths flowing in opposite directions and
connecting two Ports within adjacent Nodes.
LOGO: Logout.
Node: A collection of one or more Ports identified by a unique World
Wide Node Name (WW Node Name).
Port: The transmitter, receiver and associated logic at either end of a
link within a Node. There may be multiple Ports per Node. Each Port is
identified by a unique Port_ID, which is volatile, and a unique World
Wide Port Name (WW Port Name), which is unchangeable. In this document,
the term "port" may be used interchangeably with NL_Port or N_Port.
Port_ID: Fibre Channel ports are addressed by unique 24-bit Port_IDs. In
a Fibre Channel frame header, the Port_ID is referred to as S_ID (Source
ID) to identify the port originating a frame, and D_ID to identify the
destination port. The Port_ID of a given port is volatile (changeable).
The mechanisms through which a Port_ID may change in a Fibre Channel
topology are outside the scope of this document.
PLOGI: Port Login.
World Wide Port_Name (WWP_N): Fibre Channel requires each Port to have
an unchangeable WWP_N. Fibre Channel specifies a Network Address
Authority (NAA) to distinguish between the various name registration
authorities that may be used to identify the WWP_N. A 4-bit NAA
identifier, 12-bit field set to 0x0 and an IEEE 48-bit MAC address
together make this a 64-bit field.
World Wide Node_Name (WWN_N): Fibre Channel identifies each Node with a
unchangeable WWN_N. In a single port Node, the WWN_N and the WWP_N may be
identical.
APPENDIX - B
B.1 RELIABILITY IN CLASS 3
Problem:
Sequence ID reuse in Class 3 can conceivably result in missing frame
aliasing with no corresponding detection at the FC2 level.
Prevention:
This specification requires one of the following methods if Class 3 is
used.
- Continuously increasing Sequence Count (new Login Bit) - both
Rajagopal, Bhagawat, Rickard [Page 32]
Internet-Draft IP and ARP over Fibre Channel June 22 1998
sides must set When an N_Port sets the PLOGI login bit for
continuously increasing SEQ_CNT, it is guaranteeing that it
will transmit all frames within an exchange using a continuously
increasing SEQ_CNT (see description below).
- After using all SEQ_IDs (0-255) once, must start a new Exchange.
It is recommended that a minimum of 4 Exchanges be used before
an OX_ID can be reused.
- Note: If an implementation is not checking the OX_ID when
reassembling sequences, the problem can still occur. Cycling
through some number of SEQ_IDs, then jumping to a new exchange
does not solve the problem. SEQ_IDs must still be unique between
two N_Ports, even across exchanges.
- Use only single-frame Sequences.
B.2 CONTINUOUSLY INCREASING SEQ_CNT
This method allows the recipient to check incoming frames, knowing
exactly what SEQ_CNT value to expect next. Since the SEQ_CNT will not
repeat for 65,536 frames, the aliasing problem is significantly reduced.
A login bit (PLOGI) is used to indicate that a device always uses a
continuously increasing SEQ_CNT, even across transfers of sequence
initiative. This bit is necessary for interoperability with some
devices, and it provides other benefits as well.
In the FC-PH-3 [11], the following is supported:
Word 1, bit 17 - SEQ_CNT (S)
0 = Normal FC-PH rules apply
1 = Continuously Increasing SEQ_CNT
Any N_Port that sets Word 1, Bit 17 = 1, is guaranteeing that it will
transmit all frames within an exchange using a continuously increasing
SEQ_CNT. Each exchange shall start with SEQ_CNT = 0 in the first frame,
and every frame transmitted after that shall increment the previous
SEQ_CNT by one, even across transfers of sequence initiative. Any frames
received from the other N_Port in the exchange shall have no effect on
the transmitted SEQ_CNT.
[draft-ietf-ipfc-00.txt]
[This INTERNET DRAFT expires on Dec 22, 1998]
Rajagopal, Bhagawat, Rickard [Page 33]
| PAFTECH AB 2003-2026 | 2026-04-21 03:51:28 |