Subject: [BEHAVE] Log sequencel numbers

From: Tom Taylor <[email-masked]>
Date: Thu, 19 Sep 2013 06:48:50 -0400

Dave Thaler suggested in his review or at the meeting that we might 
assign sequence numbers to logs to allow detection of missing reports. I 
can see it as useful for all event report types, actually, But first, 
some terminology:

- an event is detected
- either an event report is generated or it is suppressed
Suppression happens as a result of a management control limiting the 
volume of a particular event type. I propose to describe such controls 
in the new Management Considerations section of the SYSLOG document.

So my proposal is that a sequence number is included in every event 
report generated. This sequence number reflects a counter that is 
carried per event type. The counter is incremented by 1 for every time 
the event is detected.

The sequence number can be used to detect missing event reports for 
critical events such as session, BIB entry, and address mapping creation 
and deletion. It can also be used to determine the number of event 
reports suppressed between successive occurrences of a given 
boundary-type event such as a threshold crossing.

Shall we add sequence numbers to IPFIX and SYSLOG? For SYSLOG, would 
they be application-specific or something that should be defined in a 
separate draft as an extension to SYSLOG?

I could also see a generic SYSLOG MIB that would, amongst other things, 
carry counters for total event detections and event reports suppressed 
by event type, but that is something I would take to the OPSAWG list.

Tom Taylor

---

From: David Harrington <[email-masked]>
Date: Thu, 19 Sep 2013 10:08:29 -0400

Hi,

See rfc5848 "signed syslog messages", which deals with assigning sequence
#s to syslog messages, for purposes of detecting missing messages.

RFC5848 takes into consideration the fact that the relationship between a
syslog sender and syslog receivers are not 1:1; syslog senders often send
some messages to one receiver and some messages to other receivers, and
receivers often accept syslog messages from  multiple senders. So the
sequence#s for received messages will not necessarily be sequential under
normal circumstances.

A specific sender uses signature blocks to sign groups of messages sent to
each specific receiver, so the receiver can tell which messages it missed.
The sender is identified by hostname, app-name, and proc-id.

Since it can be important to maintain logs for law enforcement purposes,
rfc5848 includes authentication, message integrity, and message replay as
well as sequencing and detection of missing messages. Signed syslog works
not only for the transport of messages, but also for subsequent archiving
to verify that stored message logs have not been modified. I would think
these would be really important for NAT logging.

RFC5848 also deals with issues of signing-session reboot, some recommended
approaches to reviewing logs both offline and online, and a substantial
consideration of security.

I have not looked at the techniques to determine whether they could be
adapted easily for signing ipfix and/or snmp notifications.

--
David Harrington
Syslog WG co-chair
[email-masked]
+1-603-828-1401

On 9/19/13 6:48 AM, "Tom Taylor" <[email-masked]> wrote:

>_______________________________________________
>Behave mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/behave

---

From: "Senthil Sivakumar (ssenthil)" <[email-masked]>
Date: Thu, 19 Sep 2013 14:15:04 +0000

I can understand the use of the sequence numbers in detecting missing
events, 
But I don¹t understand what the receiver/collector should do in case of a
missing event.
Are we discovering another protocol to ask for the missing events and are
we asking the
NATs to maintain event Ids so that the missing events can be replayed? I
believe this 
goes back to reliable transports and why cant we just say use a reliable
transport if 
that is the most important thing and leave it to the implementation?

Senthil

---

From: Tassos Chatzithomaoglou <[email-masked]>
Date: Thu, 19 Sep 2013 17:49:30 +0300

I could see some aid in troubleshooting.
Unless there is a connection-oriented protocol used, alerts from both source and destination are a welcome idea.

Going one step ahead, i dislike the idea of retrasmision for this matter.

--
Tassos

---

From: Tom Taylor <[email-masked]>
Date: Thu, 19 Sep 2013 12:46:48 -0400

I certainly didn't have re-transmission in mind. I meant the information 
about missing logs to be flagged at the application level, so no 
mistakes are made correlating subscribers with sessions or whatever.

I think this is a NOOP though because of David Harrington's point about 
a given collector not necessarily receiving all the logs of a given 
event type.

Tom

---

From: Tassos Chatzithomaoglou <[email-masked]>
Date: Thu, 19 Sep 2013 20:20:00 +0300

Can't sequencing be applied per collector and per event type?

The logging system of the aftr vendor i'm using now can provide differentiation per collector and some kind of differentiation per event.

--
Tassos

Tom Taylor wrote on 19/9/2013 19:46:
> I certainly didn't have re-transmission in mind. I meant the information about missing logs to be flagged at the application level, so no mistakes are made correlating subscribers with sessions or whatever.
>
> I think this is a NOOP though because of David Harrington's point about a given collector not necessarily receiving all the logs of a given event type.
>
> Tom
>

---

From: Tom Taylor <[email-masked]>
Date: Thu, 19 Sep 2013 16:46:03 -0400

For SYSLOG, then, it's as simple as suggesting the use of RFC5848. 
Though that's rather heavier than I envisioned. To get event-level 
granularity, we would have to specify an application-specific signature 
group scheme (SPRI="3").

I'd be satisfied if IPFIX just added event-specific sequence numbers as 
I originally proposed. I guess they would have to be maintained 
separately per collector to provide deployment flexibility, though it 
takes me some imagination to figure out use cases for splitting streams 
of specific event report types.

Tom Taylor

---

From: "Senthil Sivakumar (ssenthil)" <[email-masked]>
Date: Fri, 20 Sep 2013 19:42:08 +0000

>of specific event report types.

Not sure I understand, are you proposing IPFIX header carrying a sequence
number or NAT log events carrying a sequence number? As I mentioned in my
Earlier email, the sequence number don¹t really provide any advantage than
just detecting something is missing. I am against the idea of adding the
sequence numbers. I would like to see strong consensus before adding the
sequence number.

Senthil

---

From: [email-masked]
Date: Mon, 23 Sep 2013 08:41:58 -0400

> De : "Senthil Sivakumar (ssenthil)" <[email-masked]>
>
> [...] the sequence number don¹t really provide any advantage than
> just detecting something is missing. 
> 
> Senthil

Hi Senthil. 

<op-hat on>
The operational need, IMHO is exactly that: "proving that 
something is missing" or not. You don't actually need to know what
was missing, just that something was lost and ideally for how much 
time. 

When an operator is asked to provide CGN logs to law enforcement, 
it may be asked to prove in court that the logs are complete for 
the covered period. Having message numbering is one light-weight
way to do this without using stateful retransmissions (TCP) or
heavyweight message-signing (RFC5848). Both of these methods 
are probably too heavy for the kind of volume CGN logging involves. 

Now Simon will throw RFC2804 at me (again), rightfully, but I 
thought it would be a good idea to point out the *real* 
operational need again here.

/JF

---

From: "Reinaldo Penno (repenno)" <[email-masked]>
Date: Mon, 23 Sep 2013 12:53:00 +0000

Agree. This has been a requirements from many ISPs. When we looked into it there are some issues such as the sequence number + event need to be unique such that if you have several CGNs logging to the same server, you can tell the the one that had logs missing (or not). Assume here that many months have passed, configs have changed, etc.

From: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Date: Monday, September 23, 2013 5:41 AM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [BEHAVE] Log sequencel numbers

> De : "Senthil Sivakumar (ssenthil)" <[email-masked]<mailto:[email-masked]>>
>

Hi Senthil.

<op-hat on>

/JF

---

From: Simon Perreault <[email-masked]>
Date: Mon, 23 Sep 2013 15:09:17 +0200

> operational need again here.

/me throws RFC2804 at JFT ;)

IMHO, "we need to know that nothing is missing" is a valid technical
reason. The "...for law enforcement" part is what's invalid.

There may be reasons other than law enforcement why you might want to
know that nothing is missing. Handling abuse requests comes to mind.

Another idea: syslog over SCTP with partial reliability [RFC3758].
(almost not joking)

Simon

---

From: "ietfdbh" <[email-masked]>
Date: Mon, 23 Sep 2013 12:59:40 -0400

Hi Simon,

You have thrown in RFC2804, an Informational RFC about IETF policy on
wiretapping.
You apparently have argued that "for law enforcement" is invalid because it
violates the IETF policy on wiretapping.
I think that "law enforcement" and "wiretapping" are not the same thing. I'm
not sure that saying "law enforcement" violates rfc2804.
But that gets into semantics a lot ...

So, let me rephrase JF's use case to satisfy your concerns ...

"When an operator is asked to provide CGN logs,  it may be asked to prove
that the logs are complete for the covered period."

You might want to add some discussion of use cases, and their operational
and security requirements:
"For some anticipated use cases, such as law enforcement, the proof might be
held to a "strong security" standard, such as proof that would be able to
stand up in a court of law that the logs are complete and have not been
tampered with. 
For other use cases, such as a service provider handling an abuse request,
less robustness might be sufficient, but if an accused user takes the
provider to court, the service provider might need proof that meets a strong
security standard that the logs are complete and have not been altered.
For some use cases, even less robustness might be sufficient, such as the
use case of a CEO wanting a statement of assurance from the IT department
that relevant logs are not missing or modified, before they sign off on a
Sarbanes-Oxley statement, but if an investor sues the company for providing
misleading information, strong security might be required to support the
claim of completeness and integrity.
In a common use case - an operator utilizing the logs to debug network
problems- the security can probably be fairly lax or not enabled at all."

I recommend the completeness and integrity requirements of such use cases
should be spelled out in more detail.
The only mention of integrity in the syslog-nat-logging doc is in the
security considerations section, which says
" When logs are being recorded for regulatory reasons, preservation of
   their integrity and authentication of their origin is essential."
For something so essential, I would expect a bit more discussion of the
types of use cases, why integrity and authentication are essential, and more
detail on the relevant **technical** requirements of an integrity and/or
authentication solution.

Is integrity and authentication really essential only for "regulatory" use
cases? 
Is integrity and authentication really important, if not essential, for some
non-regulatory use cases?
How about when handling an abuse case? 
How about detecting an intruder attack?
How about protecting a user's privacy, given archived logs of information
such as subscriber site, hostname, devID, etc.?
What security requirements should be met for logs "in transit" and "at
rest"?

These questions apply not only to syslog, of course. The security
requirements might apply to any logging mechanism.

David Harrington
[email-masked]
+1-603-828-1401

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Simon Perreault
> Sent: Monday, September 23, 2013 9:09 AM
> To: [email-masked]

---

From: "Senthil Sivakumar (ssenthil)" <[email-masked]>
Date: Mon, 23 Sep 2013 19:35:57 +0000

RFC 7011, obsoleting rfc 5101, IPFIX header has this field

Sequence Number

      Incremental sequence counter modulo 2^32 of all IPFIX Data Records
      sent in the current stream from the current Observation Domain by
      the Exporting Process.  Each SCTP Stream counts sequence numbers
      separately, while all messages in a TCP connection or UDP session
      are considered to be part of the same stream.  This value can be
      used by the Collecting Process to identify whether any IPFIX Data
      Records have been missed.  Template and Options Template Records
      do not increase the Sequence Number.

Wouldn’t that suffice?

Thanks

Senthil

From: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Date: Monday, September 23, 2013 8:41 AM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [BEHAVE] Log sequencel numbers

Hi Senthil.

<op-hat on>

/JF

---

From: [email-masked]
Date: Mon, 23 Sep 2013 21:54:31 -0400

>       do not increase the Sequence Number.

This seems to answer Tom's original point about IPFIX. To Tom's 
defense, that document is a week old, it was published on the 15th, 
three days before the point was raised. I guess he can be excused
for not reading it within that timeframe. 

This text seems to fit the requirement for IPFIX. As long as any gap 
in continuity can be detected it should be fine. The point is still
open for syslog though. Having such a sequence number in syslog 
messages would be useful. 

/JF

---

From: Simon Perreault <[email-masked]>
Date: Tue, 24 Sep 2013 10:23:14 +0200

> violates the IETF policy on wiretapping.

(I'd prefer to keep this discussion impersonal. Even though you're
replying to me personally, the RFC2804 argument is constantly used by
many in this working group and others against any proposal facilitating
"legal intercept", "law enforcement", "wiretapping", etc.)

> But that gets into semantics a lot ...

We're clearly in RFC2804 territory when we talk about shipping CGN logs
to authorities:

   An Internet equivalent of call tracing by means of accounting logs
   (sometimes called "pen registers") that is a feature of the telephone
   network is also wiretapping by this definition, since the normal
   expectation of the sender is that the company doing the accounting
   will keep this information in confidence.

Other than that, I like your proposals very much and would agree with
your proposed text. As you have shown, we can amply justify the
technical requirements without referring to wiretapping.

Thanks,
Simon

---

From: Tom Taylor <[email-masked]>
Date: Tue, 24 Sep 2013 04:37:33 -0400

>     will keep this information in confidence.

...

My personal work-around is "regulatory requirements".

A bit of history: it was actually my WG (Megaco, media gateway control) 
that triggered the RAVEN discussion. While that went on, we went about 
our business, since room was full of telco operators and vendors. We had 
justification for forking media streams that went beyond legal 
intercept, so we just did it in the protocol.

Tom