“Fibre Channel (FC) is a high-performance bidirectional point-to-point serial data channel. The Fibre Channel standard was formulated by the T11 Standards Committee (a technical committee under the National Information Technology Standardization Committee of the United States), which is a computer communication protocol designed to meet the increasing requirements for high-performance information transmission.
Author: Zhang Yudong
Fibre Channel (FC) is a high-performance bidirectional point-to-point serial data channel. The Fibre Channel standard was formulated by the T11 Standards Committee (a technical committee under the National Information Technology Standardization Committee of the United States), which is a computer communication protocol designed to meet the increasing requirements for high-performance information transmission. FC concentrates the advantages of IO channels and networks. It not only supports the bandwidth and reliability required by IO channels, but also supports the flexibility and connectivity of network technology, making it possible to run today’s popular channel standards and network protocols on the same physical interface. possible. At present, FC has been regarded as a major networking standard for the unified network of avionics in the future. This article discusses a scheme of using system-on-chip (SOC) technology to design FC protocol chips, and analyzes the characteristics of SOC design methods and their differences with traditional embedded system design methods, which will be used in the design of avionics systems in the future The promotion of the use of SOC technology has laid the foundation.
Introduction to FC working principle
The physical media supported by Fibre Channel include optical fiber, twisted pair, coaxial cable, etc., which are collectively referred to as optical fiber in this article. Physically, FC can be regarded as the connection of multiple communication points called N ports. These N ports can be connected through a switching network and form an arbitrated loop through a hub, or they can be connected through a point-to-point link. As shown in Figure 1, the FC protocol can be divided into a series of functional levels, each functional level is briefly described as follows.
Figure 1 Hierarchical structure diagram of FC
FC-0 defines the physical characteristics of interfaces and media, and specifies the optical and electrical parameters of transceivers and various physical media. According to different implementation devices, FC can have different data transmission rates: 133Mbit/s, 266Mbit/s, 530Mbit/s, 1.0625Gbit/s, etc.
FC-1 defines the encoding, decoding and transmission protocol, and it uses DC balanced 8b/10b code. An 8-bit byte is encoded into 10 bits for transmission, and then decoded at the receiving end. A part of unused code points with special characteristics are used to form special characters to form an ordered set of signaling and frame description.
The FC-2 layer is the signal transmission protocol layer. It specifies the rules for data transmission, provides a transmission mechanism for data blocks to be transmitted from one port to the next, and defines the functions and equipment that can be used by FC-4. FC-4 can only use a subset of them. This layer describes the following concepts:
(1) Node and N port and corresponding identifier;
(2) Communication model;
(3) Topological structure;
(4) Service category;
(5) Universal switching network model;
(6) FC-2 building block and system structure;
(7) Frame format;
The FC-3 layer provides the required general services for some advanced features, such as:
(1) Classification: Use several N ports in parallel to increase the bandwidth so that a single message can be transmitted through multiple connections.
(2) Query group: enable more than one port to respond to the same alias address. This service improves efficiency by reducing the chance of contacting busy N ports.
(3) Multicast: Send a transmission to multiple target ports, including all N ports (broadcast) on a switching network, or only to certain N ports on the switching network.
The FC-4 layer is the highest layer of the FC protocol, which specifies the mapping from the upper layer protocol to the FC protocol. The currently mapped protocols are: Small Computer System Interface (SCSI), Intelligent Peripheral Interface 3 (IPI-3), High Performance Parallel Interface (HIPPI), INTERNET Protocol (IP), IEEE802.2, single-byte command code set mapping (SBCCS). In addition, FC-AE also describes an upper layer protocol mapping based on 1553B.
Physical model of FC-PH
The FC path is physically composed of at least 2 nodes. Each node can be composed of multiple N ports, and each N port provides the functions of FC-0, FC-1, and FC-2. FC-3 is optional, it provides common services for multiple N ports and FC-4. The composition of FC nodes is shown in Figure 2.
Figure 2 Composition of FC nodes
Each port consists of a pair of optical fibers, one for input and one for output. The pair of optical fibers and transceivers that transmit in opposite directions form an FC link to complete data transmission.
FC-2 layer frame format
The frame format of the FC-2 layer is shown in Figure 3.
The start of frame delimiter SOF is an ordered set of 4 8b characters, which have different code words according to different frame types; frame header, 24 8b characters, see Table 1 for details; data field, 0C2112 8b characters; CRC, 4 8b characters, it checks the frame header part and the data field part, its coding polynomial is X32+X26+X23+X22+X16+X12+X11+X10+X8+X7+X5+X4+X2+ X+1; The end-of-frame delimiter EOF is an ordered set of 4 8b characters, with different codewords according to different frame types.
Design overview of FC protocol chip
The function of the FC chip is to realize the FC-PH in Figure 1, that is, the FC-2, FC-1, and FC-0 layer protocols; each FC chip supports 2 N ports, which can form the FC node as shown in Figure 2. ; Supported data transfer rate is 1.0625Gbits/s.
Design considerations based on SoC
There are similarities between SoC-based chip design and embedded module board design, but SoC design cannot be simply understood as the miniaturization of embedded module boards. The main differences are as follows:
(1) The design of the embedded module board is realized by using the existing CPU chip and peripheral chip, while the SoC design is realized by the CPU core and various functional module cores.
(2) It can be seen from the above that the design of the embedded module board is carried out in the framework of rules that have been set. These rules are the manuals for various chips. SoC design can be designed according to the design requirements, and then design the rules by yourself. Various functional module cores to meet functional requirements and optimize resource utilization.
(3) The design of the embedded module board, due to the limitation of the CPU chip pins, the peripheral chips are mostly hung on an external bus of the CPU, it is inevitable that there will be bus contention, and the SoC design principle can work in parallel within the chip There is no limit to the number of buses. This feature should be fully utilized, and the system structure should be reasonably determined to avoid bus contention.
(4) The design of embedded module board has a fixed pattern for the design of software and hardware, and the SOC design must first determine the intersection of software and hardware according to design requirements, and determine the rules of software and hardware coordination, in order to achieve the optimization of resource utilization the goal of.
First of all, we must determine the intersection of software and hardware. The intersection of software and hardware is placed on the “sequence” of the FC-2 layer. Because the “sequence” is responsible for the reception or transmission of the entire block of data, when the length of the data block exceeds the data length shown in Figure 3, it must be divided into several related data frames for transmission. The correlation of the data frame is shown in the frame header shown in Table 1, and only the sequence count field SEQ_CNT has changed. Therefore, the software will be responsible for providing part of the information and data of the frame header, and the hardware will be responsible for data framing and subsequent work. Secondly, when designing the chip’s architecture, it is necessary to make full use of the parallel working bus inside the chip. Finally, in order to facilitate the test of this chip, for the upper interface ULP, first consider using the PCI bus to realize the FC network card based on the PCI interface. According to actual needs in the future, ULP will design on demand.
The structure of the chip
According to the discussion in Section 3.2, the block diagram of the FC protocol chip is shown in Figure 4. The interface of the FC protocol chip to the upper layer (ULP) adopts the PCI interface, which is implemented by the built-in PCI-TARGET core of the chip.
Figure 4 Block diagram of FC protocol chip
The dashed box in Figure 4 constitutes the N port of the FC, and there are 2 N ports in the entire chip. Each FC-N port is composed of two layers: “FC Transceiver Channel” and “FC Frame Transceiver Controller”. The “FC transceiver channel” cooperates with the photoelectric interface module to realize the function of sending and receiving data frames in the FC-0 layer, FC-1 layer and FC-2 layer. This layer provides an on-chip FIFO channel for the upper-layer interface to send and receive data frames. “FC Frame Transceiver Controller” realizes data frame packing, unpacking and error detection of FC-2 layer. In addition, the FC-2 layer sequence and exchange protocol are realized through the software module embedded in the chip; at the same time, the ULP interface is provided. The FC channel, frame transceiver controller, CPU core and ULP interface are discussed separately below.
Design of FC transceiver channel module
This module mainly completes the sending and receiving of FC-2 layer frames, including: completing the connection between SERDES (serial-parallel conversion module) and the optical interface, completing the FC-1 layer 8b/10b encoding and decoding, and the FC-2 layer frame-level CRC Decoding of checksum FC ordered set codes; at the same time, the built-in memory is used to form a buffer for data exchange between the FIFO and the back-end module of the FC chip.
This module is divided into “FC sending channel” and “FC receiving channel”, as shown in Figure 5. Provide a variety of self-loop test paths between the transceiver channels (not shown in the figure).
“FC sending channel” is mainly composed of the following units:
(1) Parallel-serial conversion unit;
(2) Send clock phase locked loop;
(3) 8b/10b coding unit;
(4) CRC check generation unit;
(5) TX-FIFO transmission control unit;
(6) TX-FIFO data input buffer unit;
(7) Self-loop control unit.
“FC receiving channel” is mainly composed of the following units:
(1) Serial-to-parallel conversion unit;
(2) Receive clock phase-locked loop;
(3) Clock recovery unit;
(4) Synchronization detection unit;
(5) Ordered set decoding unit;
(6) 8b/10b decoding unit;
(7) CRC check unit;
(8) RX-FIFO transmission control unit;
(9) RX-FIFO data output buffer unit;
(10) Self-loop control unit.
Figure 5 FC channel structure block diagram
Definition of data receiving and sending buffer
The definitions of the data buffer TX-FIFO and RX-FIFO are shown in Table 2: The sending buffer on the way is 33 bits, the lower 32 bits are the sent data, and the 32nd bit is used to indicate whether the current codeword is an ordered set code. The receiving buffer is 36 bits, the lower 32 bits are the received data, and the upper part stores the CRC check result of the FC-2 frame, the decoding check of the 8b/10b code, and other information.
5.1 Function overview
There are two “FC frame transceiver modules and their data buffers” in the FC chip, which implement frame packing and unpacking in the FC-2 layer protocol. It is the intersection of software and hardware of the entire system and the core of the entire system. The key is to formulate relevant design rules.
5.2 Rules for the use of data buffers
According to the principle of parallel operation of multiple buses, four independent data buffers are set up, and dual-port RAM is used to exchange data between the CPU and the frame transceiver. The rules for using the receiving and sending data buffer are as follows:
(1) Send the buffer TXBUF. According to the FC-2 frame format shown in Figure 3, this buffer is used to store the frame header and valid data. The first 24 bytes are the frame header area where the frame header is permanently stored, and the back is the valid data area. When sending, the module TXF-CTL fetches the frame header information from the frame header area to send, and continuously fetches and sends data from TXBUF according to the sending start address (TXOFFSET) and the number of sending (TXCOUNT) in the register. Here, only one frame header area is set, considering that the frame header has correlation in the “sequence”, and there is no switching of multiple areas in one frame header area, which simplifies the hardware implementation.
(2) Receive buffer RXBUF. The buffer adopts a circular queue method, and the module RXF-CTL writes data sequentially, and the data storage sequence is shown in the FC-2 frame format shown in Figure 3. After finishing 1 frame of data reception, the module RXF-CTL should indicate in the register the start address (RXOFFSET) of the current frame in RXBUF and the received number (RXCOUNT). After the CPU reads the register RXCOUNT, this register is automatically cleared. The above-mentioned usage rules are the main points of SoC design of the whole system.
The sending process of the data is sent:
(1) CMD_RESET. The module TXF-CTL resets and enters the idle state.
(2) CMD_TEST: The CPU directly controls the data transmission, and the module TXF-CTL enters the idle state.
(3) CMD_START_TX1: In the idle state and both CMD_TEST and CMD_START_TX2 are invalid, it enters the state of sending data by frame, and the module TXF-CTL sends out the SENDING signal. The module TXF-CTL first sends the ordered set code lDLE (K28.5D21.4D21.5D21.5) to the TX-FIFO of the FC sending channel; sends the corresponding SOF according to the SOF code register; then fetches the data from the frame header area of TXBUF Send sequentially; later, if the data is fetched from TXBUF, the corresponding data will be sent from TXBUF according to the register sending start address (TXOFFSET) and the number of sending (TXCOUNT); after sending the valid data, send it according to the EOF pattern register Corresponding EOF; finally send an ordered set code IDLE (K28.5D21.4D21.5D21.5). After one frame of data is sent, the SENDING signal is cleared, CMD_START_TX1 is cleared, the interrupt signal INTTXi is sent, and then it returns to the idle state. Only CMD_RESET can interrupt the sending process.
(4) If the data is in ULP-FIFO, read the number of transmissions (TXCOUNT), take the number from ULP-FIFO and send it, if the number of transmissions (TXCOUNT) is greater than the length specified by the FC-2 frame, it will be 2112 bytes The length of the frame is sent in multiple frames. After each frame is completed, the frame count value in the frame header is automatically modified, and the corresponding SOF and EOF delimiters are automatically selected; if the ULP-FIFO is empty during the transmission process, the frame of this frame is automatically ended Send and start the timeout timer, and wait for the data in the ULP-FIFO. When the data specified by the number of transmissions (TXCOUNT) has been sent or timed out, the SENDING signal will be cleared, CMD_START_TX1 will be cleared, the interrupt signal INTTXi will be sent, and then it will return to the idle state. Only CMD_RESET can interrupt the sending process.
(5) CMD_START_TX2: In the idle state when CMD_START_TX2 is valid, the module TXF-CTL is in order
The definition of the set code register and the ordered set code parameter register sends the ordered set code.
Data receiving process
There are two ways to receive data: directly controlled by the CPU; data is received by RXF-CTL according to the FC-2 frame format. The CPU can send commands to the module RXF-CTL through the register of the FC-2 frame transceiver controller to control data transmission:
(1) CMD_RESET. The module TXF-CTL resets and enters the idle state.
(2) CMD_TEST: The CPU directly controls the data reception.
(3) CMD_START_RXi: In idle state and CMD_TEST is invalid, enter the state of receiving data by frame. After the module RXF-CTL receives the ordered set code SOF from the RX-FIFO of FCSET-A, it sends out the RECEIVE signal to indicate that the reception of 1 frame of data has started. The module RXF-CTL writes the received data into the circular buffer RXBUF in turn. After receiving any ordered set code, 1 frame of data reception ends, and the RECEIVE signal is cleared. Update the register receiving start address (RXOFFSET) and the received number (RXCOUNT). If the received CRC code is invalid or the last ordered set code is not EOF, then the receive error (RXERR) is set to ‘1’, otherwise the INTRXi interrupt signal is sent to indicate that 1 frame of correct data has been received. Continue to receive the data of the next frame. In the process of receiving 1 frame of data, only CMD_RESET can be interrupted; if it has not yet begun to receive 1 frame of data, CMD_TEST can make it return to the idle state. The CPU reads the registers RXOFFSET, RXERR and RXCOUNT to know the address and number of the received data and whether the data frame is valid.
In addition, CMD_START_Rxi is also used to control the RX-FIFO controller in the FC receiving channel. CMD_START_Rx1 controls the FC receiving channel to store the received data in the RX-FIFO according to the FC-2 frame; CMD_START_Rx2 controls the FC receiving channel to receive all data (except for the continuous IDLE ordered set code) and stores it in the RXFIFO.
Built-in CPU configuration and ULP design
The built-in CPU completes the FC-2 layer sequence and exchange protocol through the embedded software, and realizes the communication between the ULP and the upper layer at the same time.
Configuration of CPU peripherals
The peripherals and bus interfaces configured by the CPU are defined as follows:
(1) Two RS232 serial ports.
(2) Watchdog timer: WATCHDOG-TIMER.
(3) Two timers for FC-2 layer communication: FCTIMER1, FC-TIMER2.
(4) Receive the PIO of two FC-2 frame transceiver controller interrupts: INTFRAME-PIO. Two PIOs that accept ordered set decoding from RX-FIFO: ORDERSET-PIO1, ORDERSETPIO2. Two internal bus interfaces, used to operate two FC-2 frame transceiver controllers. The registers of the FC-2 frame transceiver controller can be defined according to the needs of operating functions.
(5) Four independent internal bus interfaces connected to the receiving and sending data buffers.
(6) Three independent memory bus interfaces: dual-port RAM, program memory and data memory.
(7) ULP-RXFIFO interface for exchanging data with ULP.
(8) ULP-PIO interface for handshake with ULP.
ULP data channel
As shown in Figure 4, the FC chip provides two data channels for ULP: dual-port RAM and FIFO. Dual-port RAM channel: It can be divided into several partitions according to the needs of ULP, and ULP can access data to it through the PCI interface. The built-in CPU of the FC chip reads data from the dual-port RAM, puts it in TXBUF, and then starts the “frame sending module” to package and send; when receiving data, the CPU reads the data from RXBUF and stores it in the corresponding data partition of the dual-port RAM In, notify ULP to fetch data. This kind of data channel is more effective for messages that need to be confirmed and retransmitted if errors occur.
FIFO channel: ULP writes data to ULP-TXFIFO through PCI interface, CPU sets “frame sending module” to fetch data from ULP-TXFIFO, and “frame sending module” fetches data from ULP-TXFIFO when data is packed; When receiving data, the CPU reads data from RXBUF and stores it in ULP-RXFIFO. This kind of data channel is more effective for messages with high real-time requirements such as video data.
The basic operation function of FC chip embedded software
The embedded software of the FC chip must complete the FC-2 layer switching, sequence, frame sending and receiving, error control, flow control and other functions. The basic operation functions they need to call are listed in Table 3. These basic operation functions are functions related to the hardware platform, and other functions made on the platform will have nothing to do with the hardware platform. In this way, it is conducive to the collaborative development of software and hardware and isolate errors from each other. This is another important principle of SOC design.
At present, high-end FPGAs have embedded transceivers, phase-locked loops, and a large amount of memory that are suitable for high-speed transmission. The FC protocol chip discussed in this article can implement prototype prototypes on this type of FPGA. Fibre Channel will be widely used in the future avionics unified network due to its high transmission speed and good compatibility. The FC protocol chip discussed in this article will lay the material foundation for this application. This article takes the design of FC protocol chip as an example, introduces the key points of thinking based on SoC design and the basic principles of SoC design, which will help promote the use of SoC technology in the design of avionics systems.