Subject: [OPSAWG] Syslog message to Remote Rerver

From: "Aditya Dogra (addogra)" <[email-masked]>
Date: Thu, 21 Feb 2013 16:25:29 +0000

Hi All ,

Currently syslog messages collected locally on the network device are transmitted to the remote syslog servers as per RFC 5424 (UDP protocol used for transmission) and RFC 3195 (TCP protocol used for transmission)

However, we have observed that increasingly, customers are using syslog messages archived in the remote server for business logic .

In some networks, it is possible that some of the syslog messages may be dropped due to link failure or other network conditions.
However, the customers are expecting much higher resiliency for the syslog messages.

The questions we seek clarification are:

a)         What are the expectations from the external syslog delivery?

b)         Should we rely on syslog's alone ? Please note that SNMP traps functionality for network management is also there.?

Your thoughts and suggestions much appreciated.

Regards,
Aditya dogra

---

From: Christopher LILJENSTOLPE <[email-masked]>
Date: Sun, 24 Feb 2013 20:26:25 -0800

Greetings Aditya,

	Can I ask for a little more clarity as to what you are asking?  Are you asking the operational community for their expectations on syslog message reliability (it seems so in (a), or are you making a statement that you do not believe that the reliability is not sufficient (your earlier comments)?  Also, you mention in (b) that SNMP is there.  That is true - are you proposing that SNMP be used to augment syslog (if so, I would hazard to guess that that is already a solution that is widely deployed).

	Thank's,
	Christopher

On 21Feb2013, at 08.25, Aditya Dogra (addogra) <[email-masked]> wrote:

> _______________________________________________
> OPSAWG mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/opsawg

--  
李柯睿
Check my PGP key here: https://www.asgaard.org/~cdl/cdl.asc
Current vCard here: https://www.asgaard.org/~cdl/cdl.vcf
Check my calendar availability: https://tungle.me/cdl

---

From: "Aditya Dogra (addogra)" <[email-masked]>
Date: Mon, 25 Feb 2013 04:47:20 +0000

Hi Christopher,
      Sorry for not making my previous post clear. 
My point was since syslogs are tried up mostly with the base/OS layer , hence it comes pretty much earlier than the management plane comes up . And remote logging comes in picture when management plane comes up . Should syslog's be so reliable that we buffer them (in case of udp protocol) or maintain sessions (in case of tcp) (and maintain sessions during failover/switchovers) so that once management plane comes up , we send previous messages also. 

I am not talking on replacing syslogs with traps , and I agree it will be a hazard to purpose also .  My point was just like SNMP lies on management plane and hence is tightly coupled with the outgoing/exit interfaces , so it makes it more reliable in case of remote logging of traps . 
  
-Aditya dogra

-----Original Message-----
From: Christopher LILJENSTOLPE [mailto:[email-masked]] 
Sent: Monday, February 25, 2013 9:56 AM
To: Aditya Dogra (addogra)
Cc: [email-masked]; [email-masked]
Subject: Re: [OPSAWG] Syslog message to Remote Rerver

Greetings Aditya,

	Thank's,
	Christopher

---

From: Christopher LILJENSTOLPE <[email-masked]>
Date: Sun, 24 Feb 2013 21:10:13 -0800

Greetings,

	Ok, Aditya, that makes it more clear.  I assume you are just asking the operational community for their comments as to if we should buffer (or delay) sending syslog messages if the syslog server is not available?

	Thx,
	Christopher

---

From: "ietfdbh" <[email-masked]>
Date: Mon, 25 Feb 2013 01:54:14 -0500

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Aditya Dogra (addogra)
Sent: Thursday, February 21, 2013 11:25 AM
To: [email-masked]; [email-masked]
Subject: [OPSAWG] Syslog message to Remote Rerver

Hi All ,

Currently syslog messages collected locally on the network device are
transmitted to the remote syslog servers as per RFC 5424 (UDP protocol used
for transmission) and RFC 3195 (TCP protocol used for transmission) 

[dbh>] RFC5424 defines the IETF version of the syslog protocol message
format (not the UDP transport).

RFC5424 RECOMMENDS using a TLS-based transport (RFC5425) rather than a
UDP-based or plain-TCP-based transport for syslog.

If you use a UDP-based transport for interoperability, it should probably
follow RFC5426.

The IETF standard for syslog over UDP (RFC5426) states:

"   Network administrators and architects should be aware of the

   significant reliability and security issues of this transport, which

   stem from the use of UDP."

Note that RFC6587 (plain TCP transport for syslog) is Historic, and contains
an IESG Note:

   The IESG does not recommend implementing or deploying syslog over

   plain tcp, which is described in this document, because it lacks the

   ability to enable strong security [RFC3365].

   Implementation of the TLS transport [RFC5425] is recommended so that

   appropriate security features are available to operators who want to

   deploy secure syslog.  Similarly, those security features can be

   turned off for those who do not want them.

However, we have observed that increasingly, customers are using syslog
messages archived in the remote server for business logic .

[dbh>] If customers are using archived messages, they might want to consider
using signing syslog messages.

       RFC5848, Signed syslog,  describes a mechanism to add origin
authentication,

   message integrity, replay resistance, message sequencing, and

   detection of missing messages to the transmitted syslog messages.

Signed syslog helps ensure integrity of messages both in-transit and in
archived storage.

I think that would be a valuable feature in support of business logic.

David Harrington

[email-masked]

+1-603-828-1401

co-chair, syslog WG

In some networks, it is possible that some of the syslog messages may be
dropped due to link failure or other network conditions. 

However, the customers are expecting much higher resiliency for the syslog
messages. 

b)         Should we rely on syslog's alone ? Please note that SNMP traps
functionality for network management is also there.?

 

Regards,

Aditya dogra

---

From: "ietfdbh" <[email-masked]>
Date: Mon, 25 Feb 2013 17:16:15 -0500

Hi,

The syslog WG has been closed. I think OPSAWG is the appropriate forum for this discussion.
Please remove the syslog mailing list from the CC: list for this thread.

Thanks,
David Harrington
[email-masked]
+1-603-828-1401
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christopher LILJENSTOLPE
Sent: Sunday, February 24, 2013 11:26 PM
To: Aditya Dogra (addogra)
Cc: [email-masked]; [email-masked]
Subject: Re: [Syslog] [OPSAWG] Syslog message to Remote Rerver

Greetings Aditya,

	Thank's,
	Christopher

_______________________________________________
Syslog mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/syslog

---

From: Rainer Gerhards <[email-masked]>
Date: Tue, 26 Feb 2013 10:34:47 +0000

sorry for the late reply, have been off to a conference...

On Thu, 2013-02-21 at 16:25 +0000, Aditya Dogra (addogra) wrote:

> transmitted to the remote syslog servers as per RFC 5424 (UDP protocol
> used for transmission)

RFC5424 does NOT specify UDP transport. In fact, it does not specify any
transport at all, it just describes the format and the stack. Transport
mappings are done in

RFC5425 - TLS (TCP), the recommended protocol
RFC5426 - UDP

there is also historic RFC6587 on industry standard plain tcp, but this
is just for interoperating with legacy systems, not for new
implementation. It is strongly discouraged to use that in new systems.

>  and RFC 3195 (TCP protocol used for transmission) 
RFC3195 is a bit dated and would need to be changed to base on RFC5424.
This has not yet been done as there was no notable implementation of
RFC3195.

> 
> However, we have observed that increasingly, customers are using
> syslog messages archived in the remote server for business logic .
> 
>  
> 
> In some networks, it is possible that some of the syslog messages may
> be dropped due to link failure or other network conditions. 
> 

There is a very small window of exposure, see section 5.3 of RFC5425. I
also wrote a somewhat more elaborate blog post on this problem, which
may be useful for you:

http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html
> 
>  
> 
> b)         Should we rely on syslog's alone ? Please note that SNMP
> traps functionality for network management is also there.?

that's something that you need to answer based on your use cases and
requirements. As far as my personal experience goes, the loss potential
is very slim, and lot's of our customers use the RFC protocols to do biz
critical things. Some use other protocols in addition. 

side-note: modern-day syslogd implementations do not rely on the syslog
protocol alone. They accept input from a wide variety of sources,
including SNMP.

HTH
Rainer

---

From: William Herrin <[email-masked]>
Date: Tue, 26 Feb 2013 16:28:06 -0500

On Sun, Feb 24, 2013 at 11:47 PM, Aditya Dogra (addogra)
<[email-masked]> wrote:
> My point was since syslogs are tried up mostly with
> the base/OS layer , hence it comes pretty much earlier
> than the management plane comes up . And remote
> logging comes in picture when management plane
> comes up . Should syslog's be so reliable that we
> buffer them (in case of udp protocol) or maintain
> sessions (in case of tcp) (and maintain sessions
> during failover/switchovers) so that once management
> plane comes up , we send previous messages also.

Hi Aditya,

I have had servers fail with processes blocked on a syslogger stuck
trying to forward logs to a network syslog server that was no longer
available. Or trying to output logs to a serial console at 9600 bps.
The syslog blocks and then everything else blocks waiting for the
syslog.

The equipment's overall reliability comes _way_ before the reliable
transmission of any particular log line. I want the logger to quickly
dispose of the message and then accept the next one so that the
processes generating those logs don't

A colleague of mine has something he calls "reliable UDP". The idea
goes like this:

1. Transmit the message with a sequence number AND add it to a local
ring buffer.
2. If the receiver receives an out-of-sequence message, it requests
the retransmission of the missing sequence numbers.
3. If the sender receives a retransmission request, it examines the
ring buffer and retransmits if the message is still available.
4. The ring buffer overwrites its own tail as additional messages are
sent. If retransmission isn't requested before the message is
overwritten then the message is lost.

That sort of thing might be handy for syslog messages, but it the
logger is trying much harder than that, I think it risks getting in
the way of the much more important processes generating the logs.

Regards,
Bill Herrin

-- 
William D. Herrin ................ [email-masked]  [email-masked]
3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
Falls Church, VA 22042-3004