One document matched: draft-ietf-sipclf-format-05.xml
<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd">
<?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?>
<!-- try to enforce the ID-nits conventions and DTD validity -->
<?rfc strict="yes" ?>
<!-- items used when reviewing the document -->
<?rfc comments="no" ?>
<!-- controls display of <cref> elements -->
<?rfc inline="no" ?>
<!-- when no, put comments at end in comments section,
otherwise, put inline -->
<?rfc editing="no" ?>
<!-- when yes, insert editing marks -->
<!-- create table of contents (set it options).
Note the table of contents may be omitted
for very short documents -->
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="4"?>
<!-- choose the options for the references. Some like
symbolic tags in the references (and citations)
and others prefer numbers. -->
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes" ?>
<!-- these two save paper: start new paragraphs from the same page etc. -->
<?rfc compact="yes" ?>
<?rfc subcompact="no" ?>
<!-- end of list of processing instructions -->
<!-- Information about the document.
categories values: std, bcp, info, exp, and historic
For Internet-Drafts, specify attribute "ipr".
(ipr values are: full3667, noModification3667, noDerivatives3667),
Also for Internet-Drafts, can specify values for
attributes "iprExtract", and "docName". Note
that the value for iprExtract is the anchor attribute
value of a section that can be extracted, and is only
useful when the value of "ipr" is not "full3667". -->
<!-- TODO: verify which attributes are specified only
by the RFC editor. It appears that attributes
"number", "obsoletes", "updates", and "seriesNo"
are specified by the RFC editor (and not by
the document author). -->
<rfc category="std" docName="draft-ietf-sipclf-format-05" ipr="trust200902">
<front>
<title abbrev="Format for SIP CLF">Format for the Session Initiation
Protocol (SIP) Common Log Format (CLF)</title>
<!-- add 'role="editor"' below for the editors if appropriate -->
<author fullname="Gonzalo Salgueiro" initials="G" surname="Salgueiro">
<organization>Cisco Systems</organization>
<address>
<postal>
<street>7200-12 Kit Creek Road</street>
<city>Research Triangle Park</city>
<region>NC</region>
<code>27709</code>
<country>US</country>
</postal>
<email>gsalguei@cisco.com</email>
</address>
</author>
<author fullname="Vijay Gurbani" initials="V" surname="Gurbani">
<organization>Bell Labs, Alcatel-Lucent</organization>
<address>
<postal>
<street>1960 Lucent Lane</street>
<street>Rm 9C-533</street>
<city>Naperville</city>
<region>IL</region>
<code>60563</code>
<country>US</country>
</postal>
<email>vkg@bell-labs.com</email>
</address>
</author>
<author fullname="Adam Roach" initials="A" surname="B. Roach">
<organization>Tekelec</organization>
<address>
<postal>
<street>17210 Campbell Rd.</street>
<street>Suite 250</street>
<city>Dallas</city>
<region>TX</region>
<code>75252</code>
<country>US</country>
</postal>
<email>adam@nostrum.com</email>
</address>
</author>
<date year="2011" />
<!-- month="May" is no longer necessary note also, day="30" is optional -->
<area>Real Time Applications and Infrastructure</area>
<!-- WG name at the upperleft corner of the doc, IETF fine for individual submissions -->
<workgroup>SIPCLF</workgroup>
<abstract>
<t>The SIPCLF Workgroup has defined a common log format framework for
Session Initiation Protocol (SIP) servers. This common log format mimics
the successful event logging format found in well-known web
servers like Apache and web proxies like Squid. This document proposes
an indexed text encoding format for the SIP Common Log Format (CLF) that
retains the key advantages of a text-based format, while significantly
increasing processing performance over a purely text-based
implementation. This file format adheres to the SIP CLF data model and
provides an effective encoding scheme for all mandatory and optional
fields that appear in a SIP CLF record.</t>
</abstract>
</front>
<middle>
<section anchor="Introduction" title="Introduction">
<t>The extensive list of benefits and the widespread adoption of the
Apache Common Log Format (CLF) has prompted the development of a
functionally equivalent event logging mechanism for the Session
Initiation Protocol <xref target="RFC3261"></xref> (SIP). Implementing a
logging scheme for SIP is a considerable challenge. This is due in part
to the fact that the behavior of a SIP entity is more complex as
compared to an HTTP entity. Additionally, there are shortcomings to the
purely text-based HTTP Common Log Format that need to be addressed in
order to allow for real-time inspection of SIP log files <xref
target="I-D.ietf-sipclf-problem-statement"></xref>. Experience
with Apache Common Log Format has shown that dealing with large
quantities of log data can be very processor intensive, as doing so
necessarily requires reading and parsing every byte in the log file(s)
of interest.</t>
<t>An implementation independent framework for the SIP CLF has been
defined in <xref target="I-D.ietf-sipclf-problem-statement"></xref>.
This memo describes an indexed text file format for logging SIP messages
received and sent by SIP clients, servers, and proxies that adheres to
the data model presented in Section 8 of <xref
target="I-D.ietf-sipclf-problem-statement"></xref>. This document
defines a format that is no more difficult to generate by logging
entities, while being radically faster to process. In particular, the
format is optimized for both rapidly scanning through log records, as
well as quickly locating commonly accessed data fields.</t>
<t>Further, the format proposed by this document retains the key
advantage of being human readable and able to be processed using the
various Unix text processing tools, such as sed, awk, perl, cut, and
grep.</t>
</section>
<section anchor="Terminology" title="Terminology">
<t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in <xref
target="RFC2119"></xref>.</t>
<t><xref target="RFC3261"></xref> defines additional terms used in this
document that are specific to the SIP domain such as "proxy";
"registrar"; "redirect server"; "user agent server" or "UAS"; "user
agent client" or "UAC"; "back-to-back user agent" or "B2BUA"; "dialog";
"transaction"; "server transaction".</t>
<?rfc needLines="5" ?>
<t>This document uses the term "SIP Server" that is defined to include
the following SIP entities: user agent server, registrar, redirect
server, a SIP proxy in the role of user agent server, and a B2BUA in the
role of a user agent server.</t>
<t>The reader is expected to be familiar with the terminology and
concepts defined in <xref
target="I-D.ietf-sipclf-problem-statement"></xref>.</t>
</section>
<section anchor="Document_Conventions" title="Document Conventions">
<t>This document defines the logging syntax for the SIP CLF. This syntax
is demonstrated through the use of various examples. The formatting
described here does not permit these examples to be unambiguously
rendered due to the constraints imposed by the formatting rules for
Internet-Drafts. To avoid ambiguity and to meet the Internet-Draft
layout requirements this document uses the <allOneLine/> markup
convention established in <xref target="RFC4475"></xref>.</t>
<t>For the sake of clarity and completeness, the entire text defining
this markup convention from Section 2.1 of <xref
target="RFC4475"></xref> is quoted below:</t>
<t><list style="empty">
<t>Several of these examples contain unfolded lines longer than 72
characters. These are captured between <allOneLine/> tags. The
single unfolded line is reconstructed by directly concatenating all
lines appearing between the tags (discarding any line feeds or
carriage returns). There will be no whitespace at the end of lines.
Any whitespace appearing at a fold-point will appear at the
beginning of a line.</t>
<?rfc subcompact="yes" ?>
<t></t>
<t>The following represent the same string of bits:</t>
<t><list style="empty">
<t>Header-name: first value, reallylongsecondvalue, third
value</t>
<t></t>
<t><allOneLine></t>
<t>Header-name: first value,</t>
<t> reallylongsecondvalue</t>
<t>, third value</t>
<t></allOneLine></t>
<t></t>
<?rfc needLines="9" ?>
<t><allOneLine></t>
<t>Header-name: first value,</t>
<t> reallylong</t>
<t>second</t>
<t>value,</t>
<t> third value</t>
<t></allOneLine></t>
</list></t>
<?rfc subcompact="no" ?>
<t>Note that this is NOT SIP header-line folding, where different
strings of bits have equivalent meaning.</t>
</list></t>
<t>The IP addresses used in the examples in this document adhere to the
best practices outlined in <xref target="RFC5735"></xref> and correspond
to the documentation address block 192.0.2.0/24 (TEST-NET-1) as
described in <xref target="RFC5737"></xref>.</t>
</section>
<section anchor="Format" title="Format">
<t>The Common Log Format for the Session Initiation Protocol <xref
target="I-D.ietf-sipclf-problem-statement"></xref> defines a data model
to which this logging format format adheres. Each SIP CLF record MUST
consist of all the mandatory data model elements outlined in Section 8.1
of <xref target="I-D.ietf-sipclf-problem-statement"></xref>.</t>
<t><figure align="left" anchor="common_log_format"
title="SIP Common Log Format">
<preamble>All SIP CLF records MUST have the following
format:</preamble>
<artwork align="center"><![CDATA[
0 7 8 15 16 23 24 31
+-----------+-----------+-----------+-----------+
| Version | Record Length | 0 - 3
+-----------+-----------+-----------+-----------+
| Record Length (cont) | 0x2C | 4 - 7
+-----------+-----------+-----------+-----------+
| CSeq Pointer (Hex) | 8 - 11
+-----------+-----------+-----------+-----------+
| Response Status-Code Pointer (Hex) | 12 - 15
+-----------+-----------+-----------+-----------+
| R-URI Pointer (Hex) | 16 - 19
+-----------+-----------+-----------+-----------+
| Destination IP address:port Pointer (Hex) | 20 - 23
+-----------+-----------+-----------+-----------+
| Source IP address:port Pointer (Hex) | 24 - 27
+-----------+-----------+-----------+-----------+
| To URI Pointer (Hex) | 28 - 31
+-----------+-----------+-----------+-----------+
| To Tag Pointer (Hex) | 32 - 35
+-----------+-----------+-----------+-----------+
| From URI Pointer (Hex) | 36 - 39
+-----------+-----------+-----------+-----------+
| From Tag Pointer (Hex) | 40 - 43
+-----------+-----------+-----------+-----------+
| Call-Id Pointer (Hex) | 44 - 47
+-----------+-----------+-----------+-----------+
| Server-Txn Pointer (Hex) | 48 - 51
+-----------+-----------+-----------+-----------+
| Client-Txn Pointer (Hex) | 52 - 55
+-----------+-----------+-----------+-----------+
| Optional Fields Start Pointer (Hex) | 56 - 59
+-----------+-----------+-----------+-----------+
| 0x0A | | 60 - 63
+-----------+ +
| Timestamp | 64 - 67
+ +-----------+
| | 0x2E | 68 - 71
+-----------+-----------+-----------+-----------+
| Fractional Seconds | 0x09 | 72 - 75
+-----------+-----------+-----------+-----------+
| Flags Field | 76 - 79
+-----------+-----------+-----------+-----------+
|Flag (cont)| 0x09 | | 80 - 83
|-----------+-----------+ |
| |
| |
| Mandatory Fields (variable length) |
| |
| |
+-----------+-----------+-----------+-----------+
| 0x09 | Tag | 0x40 |\
+-----------+-----------+-----------+-----------+ \
| Vendor-ID | \
+-----------+-----------+-----------+-----------+ \
| Vendor-ID (cont) | \ Repeated
+-----------+-----------+-----------+-----------+ \ as many
| 0x2C | Length (Hex) | > times as
+-----------+-----------+-----------+-----------+ / necessary
| Len (cont)| 0x2C | | /
+-----------+-----------+ | /
| | /
| Value (variable length) | /
| |/
+-----------+-----------+-----------+-----------+
| 0x0A |
+-----------+
]]></artwork>
<postamble></postamble>
</figure></t>
<?rfc needLines="3" ?>
<t>The format presented in <xref target="common_log_format"></xref> is
for a single SIP CLF log entry. While there is no actual subdivision in
practice, this format can be logically subdivided into the following
three distinct components: <list style="empty">
<t>1. Index Pointers - The first 60-bytes of this format. This
portion is metadata, primarily composed of a list of pointers that
indicate the beginning of both the variable length mandatory and
optional fields that are logged as part of this record. These
pointers are implemented as a mechanism to improve processing of
these records and to allow a reader to expeditiously skip directly to
the desired field without unnecessarily going through the entire
record. This logical subdivision within the SIP CLF format will be
referenced in this document with the <IndexPointers> tag. A 0x0A (LF character) delimits <IndexPointers> from the next logical grouping.</t>
<t>2. Mandatory Fields - The next logical grouping in this format is
a tab (0x09) delimited listing of the mandatory fields as described in
Section 8.1 of <xref
target="I-D.ietf-sipclf-problem-statement"></xref> and in the order
listed in <IndexPointers>. This logical subdivision within the
SIP CLF format will be referenced in this document with the
<MandatoryFields> tag.</t>
<t>3. Optional Fields - The last logical component MAY be present as
it is an OPTIONAL extension to the SIP CLF format. Its purpose is to
provide flexibility to the developer of this SIP CLF to log any
desired fields not included in <MandatoryFields>. This
includes SIP bodies and any vendor-specific extensions. This logical
subdivision within the SIP CLF format will be referenced in this
document with the <OptionalFields> tag.</t>
</list></t>
<t><figure align="left" anchor="CLF_logical_structure"
title="Logical Structure of the SIP CLF Record">
<preamble>This logical structure of the SIP CLF record format can be
graphically represented as shown in <xref
target="CLF_logical_structure"></xref> below:</preamble>
<artwork align="center"><![CDATA[
<IndexPointers>
<MandatoryFields>
<OptionalFields>
]]></artwork>
<postamble></postamble>
</figure></t>
<t>Note that <xref target="common_log_format"></xref> and <xref
target="CLF_logical_structure"></xref> plus the terminating line-feed (0x0A) at the end of the SIP CLF record are different representations of the same format but are functionally equivalent. The representation of this format is a two line record where the <IndexPointers> metadata is on one line and the actual data like <MandatoryFields> and
<OptionalFields> (if present) is on another.</t>
<t><?rfc needLines="2" ?> In the following sections note that
indications of "hexadecimal encoded" indicate that the value is to be
written out in human-readable base-16 numbers using the ASCII characters
0x30 through 0x39 ('0' through '9') and 0x41 through 0x46 ('A' through
'F'). Similarly, indications of "decimal encoded" indicate that the
value is to be written out in human readable base-10 number using the
ASCII characters 0x30 through 0x39 ('0' through '9'). In both encodings,
numbers always take up the number of bytes indicated, and are padded on
the left with ASCII '0' (zero) characters to fill the entire space.</t>
<?rfc needLines="40" ?>
<section anchor="IndexPointers" title="Index Pointers">
<t><figure align="left" anchor="index_pointers" title="Index Pointers">
<preamble>The <IndexPointers> portion of the SIP CLF record
(shown in <xref target="index_pointers"></xref>) is a 60-byte
header that indicates metadata about the record.</preamble>
<artwork align="center"><![CDATA[
0 7 8 15 16 23 24 31
+-----------+-----------+-----------+-----------+
| Version | Record Length | 0 - 3
+-----------+-----------+-----------+-----------+
| Record Length (cont) | 0x2C | 4 - 7
+-----------+-----------+-----------+-----------+
| CSeq Pointer (Hex) | 8 - 11
+-----------+-----------+-----------+-----------+
| Response Status-Code Pointer (Hex) | 12 - 15
+-----------+-----------+-----------+-----------+
| R-URI Pointer (Hex) | 16 - 19
+-----------+-----------+-----------+-----------+
| Destination IP address:port Pointer (Hex) | 20 - 23
+-----------+-----------+-----------+-----------+
| Source IP address:port Pointer (Hex) | 24 - 27
+-----------+-----------+-----------+-----------+
| To URI Pointer (Hex) | 28 - 31
+-----------+-----------+-----------+-----------+
| To Tag Pointer (Hex) | 32 - 35
+-----------+-----------+-----------+-----------+
| From URI Pointer (Hex) | 36 - 39
+-----------+-----------+-----------+-----------+
| From Tag Pointer (Hex) | 40 - 43
+-----------+-----------+-----------+-----------+
| Call-Id Pointer (Hex) | 44 - 47
+-----------+-----------+-----------+-----------+
| Server-Txn Pointer (Hex) | 48 - 51
+-----------+-----------+-----------+-----------+
| Client-Txn Pointer (Hex) | 52 - 55
+-----------+-----------+-----------+-----------+
| Optional Fields Start Pointer (Hex) | 56 - 59
+-----------+-----------+-----------+-----------+
]]></artwork>
<postamble></postamble>
</figure></t>
<?rfc needLines="10" ?>
<t>The fields that make up <IndexPointers> are described
below:</t>
<t><list style="hanging">
<t hangText="Version (1 byte):">0x41 for this document;
hexadecimal encoded.</t>
<t hangText="Record Length (6 bytes):">Hexadecimal encoded total
length of this log record, including "Version", "Record Length",
"Flags" fields and terminating line-feed.</t>
</list> <?rfc subcompact="no" ?></t>
<t>Bytes 8 through 55 contain hexadecimal encoded pointers that point
to the starting location of each of the variable-length mandatory
fields. Note that there are no delimiters between these pointer values
-- they are packed together as a single, 52-character hexadecimal
encoded string. The "Pointer" fields indicate absolute byte values
within the record, and MUST be >=82. They point to the start of the
corresponding value within the <MandatoryFields> portion. A
description of each of the mandatory fields that these pointer values
point to can be found in <xref target="MandatoryFields"></xref>.</t>
<t><list style="hanging">
<t hangText="Optional Fields Start Pointer:">This final pointer
indicates the location within the SIP CLF record where the
OPTIONAL group of <OptionalFields> begin, if present. The
"Optional Fields Start Pointer" points to the ASCII Tab (0x09)
character for the first entry in the <OptionalFields>
portion. If the OPTIONAL group of <OptionalFields> are not
implemented, then the "Optional Fields Start Pointer" field MUST
point to the terminating line-feed (0x0A) at the end of the SIP
CLF record.</t>
</list></t>
<?rfc needLines="25" ?>
</section>
<section anchor="MandatoryFields" title="Mandatory Fields">
<t><figure align="left" anchor="mandatory_fields"
title="Mandatory Fields">
<preamble>The <MandatoryFields> portion of the SIP CLF
record is shown below:</preamble>
<artwork align="center"><![CDATA[
0 7 8 15 16 23 24 31
+-----------+-----------+-----------+-----------+
| 0x0A | | 60 - 63
+-----------+ +
| Timestamp | 64 - 67
+ +-----------+
| | 0x2E | 68 - 71
+-----------+-----------+-----------+-----------+
| Fractional Seconds | 0x09 | 72 - 75
+-----------+-----------+-----------+-----------+
| Flags Field | 76 - 79
+-----------+-----------+-----------+-----------+
|Flag (cont)| 0x09 | | 80 - 83
|-----------+-----------+ |
| |
| |
| Mandatory Fields (variable length) |
| |
| |
+-----------+-----------+-----------+-----------+
]]></artwork>
<postamble></postamble>
</figure></t>
<t>Following the pointers in <IndexPointers>, two fixed-length
fields are encoded to specify the exact time of the log entry. As
before, all fields are completely filled, pre-pending values with '0'
characters as necessary.</t>
<t><list style="hanging">
<t hangText="Timestamp (10 bytes):">Date and time of the request
or response represented as the number of seconds since the Unix
epoch (i.e. seconds since midnight, January 1st, 1970, GMT). Represented in big-endian fashion with most significant octet first from zero starting at the left, or high-order, position. Decimal encoded.</t>
<t hangText="Fractional Seconds (3 bytes):">Fractional seconds
portion of the Timestamp field to millisecond accuracy. Represented in big-endian fashion with most significant octet first from zero starting at the left, or high-order, position. Decimal
encoded.</t>
<?rfc needLines="10" ?>
<t hangText="Flags Field (5 bytes):"></t>
<?rfc subcompact="yes" ?>
<t><list style="hanging">
<t hangText="byte 1 - ">Request/Response flag</t>
<t><list style="hanging">
<t hangText="R = Request"></t>
<t hangText="r = Response"></t>
</list> <?rfc subcompact="no" ?></t>
<t hangText="byte 2 - ">Retransmission flag</t>
<?rfc subcompact="yes" ?>
<t><list style="hanging">
<t hangText="O = Original transmission"></t>
<t hangText="D = Duplicate transmission"></t>
<t
hangText="S = Server is stateless [i.e., retransmissions are not detected]"></t>
</list> <?rfc subcompact="no" ?></t>
<t hangText="byte 3 - ">Sent/Received flag</t>
<?rfc subcompact="yes" ?>
<t><list style="hanging">
<t hangText="S = Sent mesage"></t>
<t hangText="R = Received mesage"></t>
</list> <?rfc subcompact="no" ?></t>
<t hangText="byte 4 - ">Transport flag</t>
<?rfc subcompact="yes" ?>
<t><list style="hanging">
<t hangText="U = UDP"></t>
<t hangText="T = TCP"></t>
<t hangText="S = SCTP"></t>
</list> <?rfc subcompact="no" ?></t>
<t hangText="byte 5 - ">Encryption flag</t>
<?rfc subcompact="yes" ?>
<t><list style="hanging">
<t hangText="E = Encrytpted mesage (TLS, DTLS, etc.)"></t>
<t hangText="U = Unencrypted mesage"></t>
</list> <?rfc subcompact="no" ?></t>
</list></t>
</list></t>
<t>After the "Timestamp", "Fractional Seconds" and the "Flags" fields
are the actual values for the mandatory fields specified in Section
8.1 of <xref target="I-D.ietf-sipclf-problem-statement"></xref>, which
are described below:</t>
<t><list style="hanging">
<t hangText="CSeq:">The Command Sequence header field, including
the CSeq number and method name.</t>
<t hangText="Response Status-Code:">Set to the value of the SIP
response status code for responses. Set to a single ASCII dash
(0x2D) for requests.</t>
<t hangText="R-URI:">The Request-URI in the start line (mandatory
in request), including any URI parameters.</t>
<t hangText="Destination IP address:port">The IP address of the
downstream server, including the port number. For IPv4 addresses
the port number MUST be separated from the IP address by a single
':'. IPv6 addresses are represented using the bracket notation
detailed in Section 6 of <xref target="RFC5952"></xref>. That is,
the IPv6 address enclosed in square brackets and separated from
the port number by a single ':').</t>
<t hangText="Source IP address:port">The IP address of the
upstream client, including the port number over which the SIP
message was received. For IPv4 addresses the port number MUST be
separated from the IP address by a single ':'. IPv6 addresses are
represented using the bracket notation detailed in Section 6 of
<xref target="RFC5952"></xref>. That is, the IPv6 address enclosed
in square brackets and separated from the port number by a single
':').</t>
<t hangText="To URI:">Value of the URI in the To header field.</t>
<t hangText="To Tag:">Value of the tag parameter (if present) in
the To header field.</t>
<t hangText="From URI:">Value of the URI in the From header
field.</t>
<t hangText="From Tag:">Value of the tag parameter in the From
header field.</t>
</list></t>
<t>Whilst one may question the value of the From URI in light of <xref
target="RFC4474"></xref>, the From URI, nonetheless, imparts some
information. For one, the From tag is important and, in the case of a
REGISTER request, the From URI can provide information on whether this
was a third-party registration or a first-party one.</t>
<t><list style="hanging">
<t hangText="Call-Id:">The value of the Call-ID header field.</t>
<t hangText="Server-Txn:">Server transaction identification code -
the transaction identifier associated with the server transaction.
Implementations can reuse the server transaction identifier (the
topmost branch-id of the incoming request, with or without the
magic cookie), or they could generate a unique identification
string for a server transaction (this identifier needs to be
locally unique to the server only.) This identifier is used to
correlate ACKs and CANCELs to an INVITE transaction; it is also
used to aid in forking. (See Section 9.4 of <xref
target="I-D.ietf-sipclf-problem-statement"></xref> for usage.)</t>
<?rfc needLines="10" ?>
<t hangText="Client-Txn:">Client transaction identification code -
this field is used to associate client transactions with a server
transaction for forking proxies or B2BUAs. Upon forking,
implementations can reuse the value they inserted into the topmost
Via header's branch parameter, or they can generate a unique
identification string for the client transaction. (See Section 9.4
of <xref target="I-D.ietf-sipclf-problem-statement"></xref> for
usage.)</t>
</list></t>
<t>This data MUST appear in the order listed in <IndexPointers>,
and each field MUST be present. Fields are subject the maximum SIP CLF
field size of 4096 bytes as detailed in Section 8 of <xref
target="I-D.ietf-sipclf-problem-statement"></xref> and are separated
by a single ASCII Tab character (0x09). Any Tab characters present in
the data to be written will be replaced by an ASCII space character
(0x20) prior to being logged.</t>
<t>An element will not always have an appropriate value to provide for one of these fields, even when the field is required to appear in the SIP CLF record. In such circumstances, when a given mandatory field is not present then that empty field MUST be encoded as a single horizontal dash ("-").</t>
<t>In the event that a field failed to parse it MUST be encoded as a
single question mark ("?"). If these characters are part of a sequence
of other characters, then there is no ambiguity. If the field being
logged contains only one character, and that character is the literal
"-", the implementation SHOULD insert an escaped %2D for that field in
the SIP CLF record. Similarly, if the field contains only one
character, and that character is the literal "?", the implementation
SHOULD insert an escaped %3F for that field in the SIP CLF record.</t>
</section>
<?rfc needLines="18" ?>
<section anchor="OptionalFields" title="Optional Fields">
<t><figure align="left" anchor="optional_fields"
title="Optional Fields">
<preamble>The <OptionalFields> portion of the SIP CLF record
is shown below:</preamble>
<artwork align="center"><![CDATA[
0 7 8 15 16 23 24 31
+-----------+-----------+-----------+-----------+
| 0x09 | Tag | 0x40 |\
+-----------+-----------+-----------+-----------+ \
| Vendor-ID | \
+-----------+-----------+-----------+-----------+ \
| Vendor-ID (cont) | \ Repeated
+-----------+-----------+-----------+-----------+ \ as many
| 0x2C | Length (Hex) | > times as
+-----------+-----------+-----------+-----------+ / necessary
| Len (cont)| 0x2C | | /
+-----------+-----------+ | /
| | /
| Value (variable length) | /
| |/
+-----------+-----------+-----------+-----------+
]]></artwork>
<postamble></postamble>
</figure></t>
<t>Optional fields are those SIP message elements that are not a part
of the mandatory fields list detailed in Section 8.1 of <xref
target="I-D.ietf-sipclf-problem-statement"></xref>. After the
<MandatoryFields> section, there is an OPTIONAL
<OptionalFields> group (shown in <xref
target="optional_fields"></xref>) that MAY appear zero or more times.
This <OptionalFields> group provides extensibility to the SIP
CLF. It allows SIP CLF implementers the flexibility to extend the
logging capability of the indexed-ASCII representation beyond just the
mandatory log elements described in Section 8.1 of <xref
target="I-D.ietf-sipclf-problem-statement"></xref>.</t>
<t>Logging any optional SIP elements MUST be done according to the
format shown in <xref target="optional_fields"></xref>. The location
of the start of <OptionalFields> within the SIP CLF record is
indicated by the "Optional Fields Start Pointer" field in
<IndexPointers>. After the initial Tab delimiter byte (0x09)
shown in <xref target="optional_fields"></xref>, the optional field
being logged is generally represented by the notation:</t>
<t><list hangIndent="22" style="hanging">
<t>Tag@Vendor-ID,Length,Value</t>
</list></t>
<?rfc needLines="10" ?>
<t>The optional field identifier (Tag@Vendor-ID) is composed of a two
byte Tag and an eight byte Vendor-ID (both decimal encoded) separated
by an "@" character (0x40). This uniquely identifies the optional
field being logged. The format for this identifier is loosely modeled
after the private use option used by the Syslog protocol <xref
target="RFC5424"></xref> (Note: this is the second format detailed in
Section 6.3.2 of <xref target="RFC5424"></xref>). It makes use of the
Private Enterprise Number (PEN), which provides an identifier through
a globally unique name space <xref target="PEN"></xref>. This syntax
provides the necessary extensibility to SIP CLF to allow logging of
any SIP header, body, as well as any vendor-specified SIP element.</t>
<t>Optional fields are logged according to the following two syntax
rules:</t>
<t><list counter="my_count" style="format (%d)">
<t>Vendor-ID = 00000000</t>
</list></t>
<t><list style="empty">
<t>A Vendor-ID of zero is used to log the entire SIP message,
message body, Reason-Phrase or any SIP header fields that are not
a part of the mandatory fields list detailed in Section 8.1 of
<xref target="I-D.ietf-sipclf-problem-statement"></xref>. The
following Tag values are used to identify which of these optional
elements are being logged:</t>
<?rfc subcompact="no" ?>
<t><list style="hanging">
<t
hangText="Tag = 00 - Log SIP Header Field or Reason-Phrase"></t>
<t>When logging a SIP Header Field (Tag=00), the associated
"Value" field MUST be populated by the entire header field
being logged. That is, the field-name, the associated colon
(":") and the field-value. This mechanism provides the
capability to optionally log any SIP header field by
identifying the field being logged within the "Value"
field.</t>
<t>Because the Reason-Phrase in a response is part of the
Status-Line and is not identified with a field-name, it is a
special case. In this instance, the the associated "Value"
field MUST be populated by the name "Reason-Phrase" followed
by a colon (":") and a single space (SP) between the colon and
the logged Reason-Phrase value.</t>
<t>The corresponding "Length" field includes the length of the
entire "Value" field. This includes the field-name, the colon,
and any LWS separator.</t>
<t>If an optional field occurs more than once in a SIP message
(e.g. Contact, Route, Record-Route, etc.) then each occurrence
MUST be logged separately with same Tag value.</t>
<t hangText="Tag = 01 - Log message body"></t>
<t>SIP message bodies with the following body types can be
optionally logged:</t>
<?rfc subcompact="yes" ?>
<t><list style="format (%c)">
<t>Session Description Protocol (SDP) <xref
target="RFC4566"></xref> (Content-Type:
application/sdp)</t>
<t>Extensible Markup Language (XML) <xref
target="W3C.REC-xml-20081126"></xref> payloads
(Content-Type: application/*+xml)</t>
<t>binary (Content-Type: application/{isup,qsig})</t>
<t>miscellaneous text content (Content-Type:
message/sipfrag, message/http, text/plain, ...)</t>
</list></t>
<?rfc subcompact="no" ?>
<t>When logging a message body (Tag=01), the associated
"Value" field is populated with the Content-Type itself plus
the SIP message body separated with a linear white space (LWS)
separator. In this manner, everything about all four body
types is self-described using a single tag as compared to
enumerating a separate tag for each body type. Additionally,
the corresponding "Length" field includes the SIP message
body, the length of the embedded Content-Type, and the LWS
separator between the MIME type and the body content. Note
that binary bodies would have to be byte encoded to render
them in the ASCII file.</t>
<t hangText="Tag = 02 - Log entire SIP message"></t>
<t>Logging the message body (Tag=01) or the entire SIP message
(Tag=02) MUST conform to the maximum size limitation of 4096
bytes for a SIP CLF field, as detailed in Section 8 of <xref
target="I-D.ietf-sipclf-problem-statement"></xref>. These can
be repeated multiple times to accommodate SIP messages or
bodies that exceed 4096 bytes in length.</t>
</list></t>
</list></t>
<?rfc needLines="12" ?>
<?rfc subcompact="no" ?>
<t><list counter="my_count" style="format (%d)">
<t>Vendor-ID = PEN</t>
</list></t>
<t><list style="empty">
<t>A Vendor-ID set to a vendor's own private enterprise number
from the complete current list of private enterprise numbers
maintained by IANA <xref target="PEN"></xref> is used to log any
other vendor-specified optional element of a SIP header or body.
The value of the Tag is set at the discretion of the
implementer:</t>
<?rfc subcompact="yes" ?>
<t><list hangIndent="10" style="hanging">
<t hangText="Tag = Vendor-specified tag"></t>
</list></t>
</list></t>
<?rfc subcompact="no" ?>
<t>The remaining fields in the format shown in <xref
target="optional_fields"></xref> are defined below:</t>
<t><list style="hanging">
<t hangText="Length Field (4 bytes):">Indicates the length of only
the "Value" field of this optionally logged element, hexadecimal
encoded. This length does not include the header shown in <xref
target="optional_fields"></xref>.</t>
<t hangText="Value Field (0 to 4096 bytes):">Contains the actual
value of this optional field. As with the mandatory fields, ASCII
Tab characters (0x09) are replaced with ASCII space characters
(0x20).</t>
</list></t>
<t>The following are examples of optionally logged SIP elements using
the syntax described in this section. All these examples only show the
<OptionalFields> portion of the SIP CLF record. The mandatory
<IndexPointers> and <MandatoryFields> portions of the SIP
CLF are intentionally omitted for the sake of brevity. Note that all
of these examples of optionally logged fields begin with a leading Tab
delimiter byte (0x09) that is not apparent here.</t>
<t><list counter="counter" style="format (%d)">
<t>Contact header field logged as an optional field: <figure>
<artwork><![CDATA[
Consider the SIP response:
SIP/2.0 180 Ringing
<allOneLine>
Via: SIP/2.0/UDP host.example.com;
branch=z9hG4bKnashds8;received=192.0.2.1
</allOneLine>
To: Bob <sip:bob@example.com>;tag=a6c85cf
From: Alice <sip:alice@example.com>;tag=1928301774
Call-ID: a84b4c76e66710
Contact: <sip:bob@192.0.2.4>
CSeq: 314159 INVITE
Content-Length: 0
]]></artwork>
</figure><?rfc needLines="4" ?> The Contact header field would
be logged as an optional field in the following manner: <figure>
<artwork><![CDATA[
00@00000000,001C,Contact: <sip:bob@192.0.2.4>
]]></artwork>
</figure></t>
<t>Reason-Phrase logged as an optional field: <figure>
<artwork><![CDATA[
For the same SIP response the Reason-Phrase would be logged as
an optional field in the following manner:
00@00000000,0016,Reason-Phrase: Ringing
]]></artwork>
</figure></t>
<t>SDP body to be logged as an optional field: <figure>
<artwork><![CDATA[
v=0
o=alice 2890844526 2890844526 IN IP4 host.example.com
s=-
c=IN IP4 host.example.com
t=0 0
m=audio 49170 RTP/AVP 0 8 97
]]></artwork>
</figure> This body has a Content-Type of application/sdp and is
of length of 123 bytes including all the line-feeds. When logging
this body the "Value" field is composed of the Content-Type and
the body separated by a LWS, which gives it a combined length of
139 (0x008B) bytes. This SIP body would be logged as an optional
field in the following manner: <figure>
<artwork><![CDATA[
<allOneLine>
01@00000000,008B,application/sdp v=0\r\no=alice 2890844526
2890844526 IN IP4 host.example.com\r\ns=-\r\n
c=IN IP4 host.example.com\r\nt=0 0\r\n
m=audio 49170 RTP/AVP 0 8 97\r\n
</allOneLine>
]]></artwork>
</figure> Note that the body is actually logged on a single line
and are thus captured between <allOneLine/> tags. The
line-feeds are escaped using \r\n to delimit the various lines in
the message body.</t>
<t>Codec information from the SDP body logged as an optional
field: <figure>
<artwork><![CDATA[
Consider the SIP message:
INVITE sip:bob@example.com SIP/2.0
Via: SIP/2.0/UDP host.example.com;branch=z9hG4bKnashds8
To: Bob <bob@example.com>
From: Alice <alice@example.com>;tag=1928301774
Call-ID: a84b4c76e66710
CSeq: 314159 INVITE
Max-Forwards: 70
Date: Thu, 21 Feb 2002 13:02:03 GMT
Contact: <sip:alice@host.example.com>
Content-Type: application/sdp
Content-Length: 147
v=0
o=UserA 2890844526 2890844526 IN IP4 example.com
s=Session SDP
c=IN IP4 host.example.com
t=0 0
m=audio 49172 RTP/AVP 0
a=rtpmap:0 PCMU/8000
]]></artwork>
</figure> A vendor may choose to log a SIP message element such
as the codec information from the SDP body. This Vendor-specified
SIP element would be logged as an optional field in the following
manner: <figure>
<artwork><![CDATA[
03@00032473,0014,a=rtpmap:0 PCMU/8000
]]></artwork>
</figure></t>
<t>N-th message received from a particular peer logged as an
optional field: <figure>
<artwork><![CDATA[
Perhaps a vendor wants to log that this message is the n-th
message received from a peering partner. To do so for the SIP
message shown above, the vendor would log this information as:
07@00032473,0016,1877 example.com
]]></artwork>
</figure><?rfc needLines="3" ?> Which would signify that this is
the 1,877th message from the peering partner example.com. Note
that the previous two examples showing an optionally logged
Vendor-specified SIP element use a Vendor-ID with a Private
Enterprise Number of 32473. This value has been reserved by IANA
to be used as an example PEN in documentation according to <xref
target="RFC5612"></xref>.</t>
</list></t>
</section>
</section>
<?rfc needLines="45" ?>
<section anchor="Example_Record" title="Example SIP CLF Record">
<t>The following SIP message is an INVITE request sent by a SIP client:
<figure>
<artwork><![CDATA[
INVITE sip:192.0.2.10 SIP/2.0
To: <sip:192.0.2.10>
Call-ID: DL70dff590c1-1079051554@example.com
<allOneLine>
From: "Alice" <sip:1001@example.com:5060>;
tag=DL88360fa5fc;epid=0x34619b0
</allOneLine>
CSeq: 1 INVITE
Max-Forwards: 70
<allOneLine>
Via: SIP/2.0/TCP 192.0.2.200:5060;
branch=z9hG4bK-1f6be070c4-DL
</allOneLine>
Contact: "1001" <sip:1001@192.0.2.200:5060>
<allOneLine>
Allow: INVITE,CANCEL,ACK,OPTIONS,INFO,SUBSCRIBE,NOTIFY,BYE,
MESSAGE,UPDATE,REFER
</allOneLine>
Supported: replaces,norefersub
User-Agent: Some Vendor
Content-Type: application/sdp
Content-Length: 418
v=0
o=1001 1456139204 0 IN IP4 192.0.2.200
s=-
c=IN IP4 192.0.2.200
b=AS:2048
t=0 0
m=audio 13756 RTP/AVP 0 101
a=rtpmap:0 PCMU/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=x-mpdp:192.0.2.200:13756
m=video 13758 RTP/AVP 96
a=rtpmap:96 H264/90000
<allOneLine>
a=fmtp:96 profile-level-id=420015; max-mbps=47520; max-fs=1584;
max-dpb=7680
</allOneLine>
a=x-mpdp:192.0.2.200:13758
]]></artwork>
</figure> Shown below is approximately how this message would appear
as a single record in a SIP CLF logging file if encoded according to the
syntax described in this document. Due to internet-draft conventions,
this log entry has been split into five lines, instead of the two lines
that actually appear in a log file; and the tab characters have been
padded out using spaces to simulate their appearance in a text terminal.
<figure>
<artwork><![CDATA[
A0000FE,0053005C005E006D007D008F009E00A000BA00C700EB00F500FE
<allOneLine>
0000000000.010 RORUU 1 INVITE - sip:192.0.2.10
192.0.2.10:5060 192.0.2.200:56485 sip:192.0.2.10 -
sip:1001@example.com:5060 DL88360fa5fc
DL70dff590c1-1079051554@example.com server-tx client-tx
</allOneLine>
]]></artwork>
</figure>A Base64 encoded version of this log entry (without the changes required to format it for an Internet-Draft) is shown below.
<figure>
<artwork align="left"><![CDATA[
begin-base64 644 clf_record
QTAwMDBGRSwwMDUzMDA1QzAwNUUwMDZEMDA3RDAwOEYwMDlFMDBBMDAwQkEwMEM3MDBF
QjAwRjUwMEZFCjAwMDAwMDAwMDAuMDEwICBST1JVVSAgIDEgSU5WSVRFICAgICAgICAt
ICAgICAgIHNpcDoxOTIuMC4yLjEwICAxOTIuMC4yLjEwOjUwNjAgMTkyLjAuMi4yMDA6
NTY0ODUgICAgICAgc2lwOjE5Mi4wLjIuMTAgIC0gICAgICAgc2lwOjEwMDFAZXhhbXBs
ZS5jb206NTA2MCAgICAgICBETDg4MzYwZmE1ZmMgICAgREw3MGRmZjU5MGMxLTEwNzkw
NTE1NTRAZXhhbXBsZS5jb20gICAgIHNlcnZlci10eCAgICAgICBjbGllbnQtdHgK
====
]]></artwork>
</figure>To recover the unencoded file, the Base64 text above may be passed as input to the following perl script (the output should be redirected to a file).
<figure>
<artwork><![CDATA[
#!/usr/bin/perl
use strict;
my $bdata = "";
use MIME::Base64;
while(<>)
{
if (/begin-base64 644 clf_record/ .. /-- ==== --/)
{
if ( m/^\s*[^\s]+\s*$/)
{
$bdata = $bdata . $_;
}
}
}
print decode_base64($bdata);
]]></artwork>
</figure></t>
</section>
<section anchor="Text_Tool_Considerations"
title="Text Tool Considerations">
<t>This format has been designed to allow text tools to easily process
logs without needing to understand the indexing format. Index lines may
be rapidly discarded by checking the first character of the line: index
lines will always start with an alphabetical character, while field
lines will start with a numerical character.</t>
<t>Within a field line, script tools can quickly split fields at the tab
characters. The first 12 fields are positional, and the meaning of any
subsequent fields can be determined by checking the first four
characters of the field. Alternately, these non-positional fields can be
located using a regular expression. For example, the "Contact value" in
a request can be found by searching for the perl regex
/\t0000,....,([^\t]*)/.</t>
</section>
<section anchor="Security" title="Security Considerations">
<t>This document does not introduce any new security considerations
beyond those discussed in <xref
target="I-D.ietf-sipclf-problem-statement"></xref>.</t>
</section>
<section anchor="Operational_Guidance" title="Operational Guidance">
<t>SIP CLF log files will take up substantive amount of disk space
depending on traffic volume at a processing entity and the amount of
information being logged. As such, any enterprise using SIP CLF should
establish operational procedures for file rollovers as appropriate to
the needs of the organization.</t>
<t>Listing such operational guidelines in this document is out of scope
for this work.</t>
</section>
<section anchor="IANA" title="IANA Considerations">
<t>This document does not require any considerations from IANA.</t>
</section>
<section anchor="Acknowledgements" title="Acknowledgements">
<t>The authors of this document would like to acknowledge and thank
Peter Musgrave for his support, guidance, and continued invaluable
feedback.</t>
<t>This work benefited from the discussions and invaluable input by the
various members of the SIPCLF working group. These include Brian
Trammell, Eric Burger, Cullen Jennings, Benoit Claise, Saverio
Niccolini, Dan Burnett. Special thanks to Hadriel Kaplan, Chris Lonvick,
Paul E. Jones, John Elwell for their constructive comments, suggestions,
and reviews that were critical to the formulation and refinement of this
draft.</t>
<t>Thanks to Anders Nygren for his early implementation, insight, and
reviews of the SIP CLF format.</t>
</section>
</middle>
<?rfc needLines="10" ?>
<back>
<references title="Normative References">
<?rfc include="reference.I-D.ietf-sipclf-problem-statement" ?>
<?rfc include="reference.RFC.2119" ?>
<?rfc include="reference.RFC.3261" ?>
<?rfc include="reference.RFC.5424" ?>
</references>
<references title="Informative References">
<?rfc include="reference.RFC.4474" ?>
<?rfc include="reference.RFC.4475" ?>
<?rfc include="reference.RFC.4566" ?>
<?rfc include="reference.RFC.5612" ?>
<?rfc include="reference.RFC.5735" ?>
<?rfc include="reference.RFC.5737" ?>
<?rfc include="reference.RFC.5952" ?>
<?rfc include="reference.W3C.REC-xml-20081126" ?>
<reference anchor="PEN">
<front>
<title>Private Enterprise Numbers</title>
<author fullname="" initials="" surname="IANA">
<organization>IANA</organization>
</author>
<date month="" year="2009" />
</front>
<seriesInfo name="http://www.iana.org/assignments/enterprise-numbers"
value="" />
</reference>
</references>
</back>
</rfc>
| PAFTECH AB 2003-2026 | 2026-04-24 04:07:57 |