Subject: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

From: Yakov Shafranovich <[email-masked]>
Date: Sun, 11 Jul 2010 11:53:10 -0400

This is in regards to the following draft:

http://www.ietf.org/id/draft-cai-smsc-mmsc-reporting-marf-00.txt

As per emails to the list and the SpamRep documentation at the OMA,
this draft is trying to address the gap between MARF and the OMA
SpamRep requirements. One key point to note is that the MARF model is
focused on feedback loops between the ISP and the original email
sender, while the OMA SpamRep activity is focused on the end user and
their network provider. The analogous use case in the email space is a
"report spam" button for end users which creates a report for their
ISP.

1. Section 1.2 "requirements" - this seems to repeat the main MARF
draft requirements and seems to be different from the OMA
requirements. There is no need to repeat the MARF requirements
verbatim, it may be sufficient to reference them. The OMA requirements
should be referenced, and some of the OMA SpamRep requirements which
cannot be solved with MARF should be addressed explicitly here.
Ideally this section should clearly state that this draft seeks to
address OMA requirements X and Y, but not Z.

2. There needs to be a reference in the draft to the standard that
defines SMS/MMS message formats. This is needed in order for us to see
how it differs from email. Additionally, you need to specify whether
these SMS/MMS spam report will be delivered via email. If not, this
may not fall within MARF or IETF scope of work.

3. As per the MARF draft, there is a specific format to be followed
when defining fields for the registry. You need to specify which abuse
type it references, whether it is included once or multiple times, and
whether it is historical. That format should be followed.
Additionally, ABNF would not hurt.

4. The draft keeps referencing abuse detected by the message center.
However, the OMA SpamRep requirements talk about the end user
reporting spam.

5. One of the basic principles underpinning MARF is that the data
included in section 2 of the report, should be data that cannot be
otherwise derived from the original message. Many of the fields that I
see listed seem to be ones that will be included in the original
message itself. There is no need to repeat them again in section 2 of
the report.

Some field specific comments:
6. "Message ID" - in the email world, this is automatically generated
in the headers of the MARF report itself.
7. "Message Center ID" - in the email world the sender of the MARF
report would be in the headers.
8. "Message Type" - MARF is specific for email, IM reporting has its
own formatting. You will need to create a registry or tightly define
this. Also, is this information that would be included in the original
message?
9. Fields 4.4 - 4.15 - are these included in the original message? Is
there a need to repeat them in part 2?
10. Abuse types - is there a need for all of these? Prior discussion
here seems to indicate that most of these are not necessary and some
overlap. For example, message flooding and DOS are the same.
11. Abuse detection methods - this is similar to the authentication
results header we currently discussed regarding DKIM. There needs to
be a document covering this.
12. Sections 4.18-4.23 - is any of this data included in the original
message? Also, are any of these currently used?
13. Regarding feedback types - I would recommend looking at the
possibility of defining a new feedback type called "sms" or "mobile"
and re-using as much as possible of existing MARF. For example, abuse
keywords, etc. could be specified using URIs.

Yakov

---

From: "Wenjun (Kevin) Zeng" <[email-masked]>
Date: Sun, 11 Jul 2010 12:40:32 -0500

Hi Yakov and all,

A couple of clarifications.

1. Although OMA SpamRep was originally motivated by feedback loop between
the end user and
their network provider, the actual specification only specifies two logical
entities, i.e., a SpamRep server and a SpamRep client where the client could
be UA, MTA, SMSC etc.

2. OMA SpamRep delivers Spam Reports through HTTP protocol. Is MARF
restricted to email delivery of report only?

Thanks.

-Kevin

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Yakov Shafranovich
Sent: Sunday, July 11, 2010 10:53 AM
To: [email-masked]
Subject: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

Yakov
_______________________________________________
marf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/marf

---

From: Yakov Shafranovich <[email-masked]>
Date: Sun, 11 Jul 2010 15:16:26 -0400

1. Thanks.
2. As of now, the intention of MARF is to be exchanged via email only.
It is a "son of" RFC 3462 which is "multipart/report" for
administrative messages regarding email. Of course, other transport
methods could be defined in the future.

Yakov

---

From: "Alex Bobotek" <[email-masked]>
Date: Sun, 11 Jul 2010 13:19:00 -0700

I believe that a single standard for mobile reporting (OMA and/or IETF)
should be our goal, and am very interested in seeing OMA and IETF
develop standards that do not fragment use and implementation, whether
by having one or both bodies produce applicable standard(s).

SpamRep was motivated by a desire to provide reporting from more than
end users:  to provide a standard format for mobile reporting.  Yes, it
is important to have a mobile spec that can be incorporated into phones
by device manufacturers otherwise uninvolved in anti-abuse.  However,
the SMS portion was writer to specifically support the following (and
hopefully other) classes of reporting:

*  Manual reporting from an SMS UA to a network node
*  Automated reporting from an SMS UA to a network node
*  Automated/manual reporting from an SS7 or SMPP network node such as
an SMSC
*  Intercarrier abuse notifications

SMS abuse reporting presents many challenges that are less prominent in
email, notably that there are inherently many gateways (protocol
conversions) in the delivery path, and very little backtrace information
(as in email headers).  

In drafting the SMS portion of SpamRep, I found supporting multiple
reporter classes to be a challenge, but not an impossibility.  Separate
specifications seem to be an unnecessary and disadvantageous approach.  

While contributing to OMA SpamRep, OMA contributors (myself included)
been following ARF/MARF, even using it as a baseline for many initial
SpamRep requirements and features.  Two key differences between SpamRep
and MARF are:

* SMTP vs HTTP (at least typically) for transport 
* XML vs RFC 5322 (email) style headers for report and message metadata

IMO, either transport-independence (costly in that transport features
such as encryption and authentication may be leveraged) or adaptation to
multiple transports (my preference) is desirable and entirely feasible.

The key difference is that SpamRep is XML-based while
thdraft-cai-smsc-mmsc-reporting-marf-00 draft and MARF is header-based.
At the root of this is that email works this way, while many other
communications systems use XML.  Does anyone have any suggestions on how
to drive convergence?  

Regards,

Alex

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Wenjun (Kevin) Zeng
Sent: Sunday, July 11, 2010 10:41 AM
To: 'Yakov Shafranovich'; [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

Hi Yakov and all,

1. Although OMA SpamRep was originally motivated by feedback loop
between
the end user and
their network provider, the actual specification only specifies two
logical
entities, i.e., a SpamRep server and a SpamRep client where the client
could
be UA, MTA, SMSC etc.

Thanks.

-Kevin

-----Original Message-----

Yakov

---

From: Yakov Shafranovich <[email-masked]>
Date: Sun, 11 Jul 2010 19:33:44 -0400

One of the main issues with the draft as it is, is the basic principle
of MARF as it was designed - section 2, the machine readable section,
should not contain any data that can be derived from the message
itself. Because we don't know what standard you are using for defining
SMS / MMS formats, I can't figure out whether the various message
fields are already included in the original message.

Yakov

---

From: "Alex Bobotek" <[email-masked]>
Date: Sun, 11 Jul 2010 21:08:30 -0700

Yakov,

I'm not certain whether your comment was intended to addresses the SpamRep, draft-cai-smsc-mmsc-reporting-marf-00 or both.  However, field definition and mapping is a challenge for any SMS standard.  Even in a most common type of SMSC (SMPP<---->GSM MAP), an abusive message may be received in MAP or SMPP protocols.  

The Sad Fact of SMS Mangling during (and after) Transport

The lack of a single transport protocol over the numerous hops in SMS transport has its challenges; the "original message" often does not exist.  GSM OTA-->GSM MAP-->SMPP-->SMPP-->GSM MAP-->GSM OTA gateway and protocol conversions are common.  And GSM and SMPP standards are by no means the only ones in play, nor is there necessarily any indication of the route a message may have traversed or what protocol conversions or modifications may have been performed.  Beyond this, there is segmentation/reassembly, and a UA may render SMS text into Unicode or some other format for local storage, and discard much of an original message's (perhaps GSM MAP) information.  Even where unadulterated copies of a message may exist, their fidelity may be uncertain.  For example, what was the original UDH information?  Without a complete set of GSM MAP and/or SMPP attributes and perhaps original padding, a full corpus cannot be constructed.  And simply dropping, say, a UDH header attribute in GSM MAP may substantially mangle a message.  

So how should this be handled?  Should there be different protocols for reporting from SMSCs, SMPP, GSM MAP, clients, SIP IPSM Gateways, STPs, user agents, etc.?  I think not.  This uncertainty as to the fidelity of 'corpus' is seemingly incompatible with a pragma of excluding from metadata (e.g, Section 2) anything which may be contained in message corpus.  A further complication is the lack of delivery path info in SMS (no Received headers).  SpamRep further allows for variant formats which, at times place similar information in XML and other times in corpus.  

IMHO, an applicable set of pragmas for SMS is:
*	Report what you can
	*	Report as fact what you know
	*	Report what you are told as hearsay

OMA SMS SpamRep attributes were largely extracted from GSM MAP and SMPP protocol fields.  But it is recognized that many attributes are not always available.  It is PREFERRED that accurate corpus be attached, but this isn't always possible.  In OMA SMS SpamRep there is metadata (i.e., DecodedUDIndicator and UDHAttached) that indicates whether or not, for example, the attached SMS corpus contains verbatim UD and UDH.  Another important metadata field (sometimes discernable from the message corpus) is SpamRep's 'ReportingNode' {SMS_Router,
IP-SM-GW, SMS Router, IP-SM-GW, SMS-SC, Mobile_Device, Subscriber, Email_Gateway, Unknown, Other} attribute.

OMA SpamRep SMS reporting attempts to address the myriad challenges of SMS abuse reporting discussed above, and others.  

Regards,

Alex

-----Original Message-----
From: Yakov Shafranovich [mailto:[email-masked]] 
Sent: Sunday, July 11, 2010 4:34 PM
To: Alex Bobotek
Cc: [email-masked]; [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00": XML vs Headers

Yakov

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 12 Jul 2010 00:10:24 -0400

Considering this, is it smart to shore horn SMS abuse reporting into MARF?

---

From: John Levine <[email-masked]>
Date: 12 Jul 2010 05:22:06 -0000

>restricted to email delivery of report only?

MARF defines a MIME type.  To date, it's always been sent as the body
of an email message, but HTTP also transmits MIME objects.

For that matter, since SpamRep is also a MIME type, you can send
those, at least the ones that don't ask for an immediate response, by
e-mail.

Given how little SMTP and SMS have in common, I'd be more inclined to
keep them separate and encourage places that can accept both to accept
them both by mail and HTTP.

R's,
John

---

From: "Alex Bobotek" <[email-masked]>
Date: Sun, 11 Jul 2010 23:45:00 -0700

I'm in agreement with John that MARF/Email and SpamRep/SMS are so far
apart that it would be difficult to force them to converge.  Feedback
and abuse management from a mobile SMS user agent has requirements for
which HTTP (or any request/response protocol) is more advantageous than
SMTP.  This is IMO true for any UA-based feedback, but especially so in
the mobile case.  

SpamRep is a request/response transaction protocol in which some
processing (with potential side effects) may be required prior to an
application-layer response.  As written, it is not completely compatible
with store & forward SMTP relay.  It would be awkward to mail back an
asynchronous response to a device that has limited Internet
connectivity, and some sort of extension to SMTP would likely be
necessary for handling a MIME response.  Lemonade-like optimizations
that prevent over-the-air transmission of messages already in possession
of the receiving system are also needed.  SMTP is not an ideal transport
for SpamRep.  

Emailing back protocol responses to cellular mobile devices (even if not
over port 25, and especially if not immediate) isn't an attractive
architecture.  A substantial subset of SpamRep could, in theory, operate
over a message transport protocol such as SMTP.  

So it's probably best to use SpamRep for SMS.  

Note that SpamRep does support Email abuse reporting over HTTP.  Perhaps
any overlap in this area should be discussed.

Regards,

Alex

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
John Levine
Sent: Sunday, July 11, 2010 10:22 PM
To: [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

R's,
John

---

From: "John R Levine" <[email-masked]>
Date: 12 Jul 2010 03:26:54 -0400

> any overlap in this area should be discussed.

In the ASRG, we had a robust discussion about the way one might add a spam 
button to a MUA:

http://wiki.asrg.sp.am/wiki/Adding_a_junk_button_to_MUAs

One of the open questions is how the MUA sends the report back to the 
appropriate place, which in this case is similar to the SpamRep model, the 
mail server from which the MUA picked up the mail.  HTTP is not a bad way 
to go, particularly on mobile devices where users are likely to have both 
e-mail and SMS/MMS spam to report.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 12 Jul 2010 10:22:13 -0700

> Alex Bobotek
> Sent: Sunday, July 11, 2010 11:45 PM
> To: John Levine; [email-masked]

It seems to me that SpamRep does two things: It defines a message format and it defines the interaction between the sender of the report (Client) and the report receiver (Server).  As a message format is transport-agnostic, one could use HTTP POST to send either a SpamRep message or a MARF message.

In the former case, we would simply not require the Server response when used via email.  That is, the protocol part of SpamRep doesn't make sense in the email side of things but the message format part would probably be fine.

---

From: "John R Levine" <[email-masked]>
Date: 12 Jul 2010 13:33:27 -0400

> In the former case, we would simply not require the Server response when used via email.  That is, the protocol part of SpamRep doesn't make sense in the email side of things but the message format…

I could imagine a hypothetical model in which the server would mail back 
the responses, but it's hard to see a practical application.  There's a 
reason all those servers we had 15 years ago to retrieve stuff by mail 
aren't around any more.

On the other hand, a model where an MUA does a POST to send back an ARF 
report could work well.  Most current MUAs have 90% of a web browser built 
in already to render HTML mail so I expect that it would not be a lot of 
extra programming to add.

Regards,

---

From: Douglas Otis <[email-masked]>
Date: Mon, 12 Jul 2010 11:03:18 -0700

On 7/12/10 10:33 AM, John R Levine wrote:
>> In the former case, we would simply not require the Server response 
>> when used via email.  That is, the protocol part of SpamRep doesn't 
>> make sense in the email side of things but the message format part 
>> would probably be fine.
>
> I could imagine a hypothetical model in which the server would mail 
> back the responses, but it's hard to see a practical application.  
> There's a reason all those servers we had 15 years ago to retrieve 
> stuff by mail aren't around any more.
>
> On the other hand, a model where an MUA does a POST to send back an 
> ARF report could work well.  Most current MUAs have 90% of a web 
> browser built in already to render HTML mail so I expect that it would 
> not be a lot of extra programming to add.
Why introduce yet another messaging system?  Such protocol diversity 
will likely depend upon insecure automatic configurations, where this 
new transport would add HTTP redirection vulnerabilities, and introduce 
acceptance issues when session interaction is expected, such as 
including Captcha.  From a management and security standpoint, less is more.

-Doug

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 12 Jul 2010 20:11:28 +0200

>> any overlap in this area should be discussed.

+1

> In the ASRG, we had a robust discussion about the way one might add a
> spam button to a MUA:
>
> appropriate place, which in this case is similar to the SpamRep model,
> the mail server from which the MUA picked up the mail. HTTP is not a bad
> way to go, particularly on mobile devices where users are likely to have
> both e-mail and SMS/MMS spam to report.

There is a "RID Transport" [1] draft that may have parallel facets 
with this use of HTTP.  Although that is focused on the team-to-team 
step rather than the user-to-team one, several considerations about, 
say, TLS, port numbers, and URIs may be relevant.

I take this occasion to repeat that discussing any overlapping with 
the IODEF effort is probably worth too.

-- 
[1] http://tools.ietf.org/html/draft-moriarty-post-inch-rid-transport

---

From: "J.D. Falk" <[email-masked]>
Date: Mon, 12 Jul 2010 12:33:02 -0600

+1

+1

draft-jdfalk-marf-reporting-discovery also mentions MUA submissions via HTTP, though I kept it simple: a POST of a message/rfc822 MIME part to the specified URI.  SpamRep may be a better way to go, particularly given the increasing availability of fully-featured MUAs on mobile devices.

--
J.D. Falk <[email-masked]>
Return Path Inc

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 12 Jul 2010 20:22:30 -0400

> In the former case, we would simply not require the Server response when used via email.  That is, the protocol part of SpamRep doesn't make sense in the email side of things but the message format…

I think this is a good way of looking at it. There are two separate
issues here - one is that of transport and second of the actual
format. It may be prudent to attack them separately.

The transport issue per se isn't really a big deal. We have been
talking about using non-SMTP ways to route ARF reports here and there,
so HTTP may be another way. I think that perhaps splitting that into a
separate draft may work better.

I have a bigger concern with the message format itself. The draft is
defining over 15 new headers. Without knowing how SMS / MMS and mobile
IM is formatted, it is very hard to shoehorn this into MARF.
Additionally, within the MARF model, the lowest common denominator is
the message itself - it is primae faceae evidence of the spam activity
and part 2 is only included to assist people with information not
otherwise available.

Perhaps as a way to converge SpamRep and MARF may be to focus
specifically on reporting spam email messages from mobile phones. That
would be addressable via existing draft with a new transport method
perhaps, and a few additional headers. The next step would be to
tackle IM, perhaps via additional headers already defined for XMPP,
and then SMS itself.

It would also be helpful if the people on the mobile space can explain
what the various headers are, what they do and why they are not being
included in the original spam message.

Thanks,
Yakov

---

From: "Cai, Yigang (Yigang)" <[email-masked]>
Date: Tue, 13 Jul 2010 06:25:03 -0500

Yakov,

I put 15 new parameters in the draft because most of them are useful info and spam classification which can be detected or by canalization from SMSC/MMSC. We can reduce number of new parameters, and only list some of common parameters among email, SMS/MMS, and IM.

I agree with you the transport isn't big issue. But at the beginning, we may define SMTP and HTTP as base.

Yigang

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Yakov Shafranovich
Sent: Monday, July 12, 2010 7:23 PM
To: [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

> In the former case, we would simply not require the Server response when used via email.  That is, the protocol part of SpamRep doesn't make sense in the email side of things but the message format…

I think this is a good way of looking at it. There are two separate issues here - one is that of transport and second of the actual format. It may be prudent to attack them separately.

The transport issue per se isn't really a big deal. We have been talking about using non-SMTP ways to route ARF reports here and there, so HTTP may be another way. I think that perhaps splitting that into a separate draft may work better.

I have a bigger concern with the message format itself. The draft is defining over 15 new headers. Without knowing how SMS / MMS and mobile IM is formatted, it is very hard to shoehorn this into MARF.
Additionally, within the MARF model, the lowest common denominator is the message itself - it is primae faceae evidence of the spam activity and part 2 is only included to assist people with information not otherwise available.

Perhaps as a way to converge SpamRep and MARF may be to focus specifically on reporting spam email messages from mobile phones. That would be addressable via existing draft with a new transport method perhaps, and a few additional headers. The next step would be to tackle IM, perhaps via additional headers already defined for XMPP, and then SMS itself.

It would also be helpful if the people on the mobile space can explain what the various headers are, what they do and why they are not being included in the original spam message.

Thanks,
Yakov

---

From: "Cai, Yigang (Yigang)" <[email-masked]>
Date: Tue, 13 Jul 2010 09:40:42 -0500

Yakov,

Please see my response inline.

Thanks,

Yigang

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Yakov Shafranovich
Sent: Sunday, July 11, 2010 10:53 AM
To: [email-masked]
Subject: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

As per emails to the list and the SpamRep documentation at the OMA, this draft is trying to address the gap between MARF and the OMA SpamRep requirements. One key point to note is that the MARF model is focused on feedback loops between the ISP and the original email sender, while the OMA SpamRep activity is focused on the end user and their network provider. The analogous use case in the email space is a "report spam" button for end users which creates a report for their ISP.
[Yigang] SMS/MMS abuse somehow should be covered either by OMA or MARF. OMA SpamRep currently seems for a report from mobile device to server. MARF is a report from a server to a server. 

1. Section 1.2 "requirements" - this seems to repeat the main MARF draft requirements and seems to be different from the OMA requirements. There is no need to repeat the MARF requirements verbatim, it may be sufficient to reference them. The OMA requirements should be referenced, and some of the OMA SpamRep requirements which cannot be solved with MARF should be addressed explicitly here.
Ideally this section should clearly state that this draft seeks to address OMA requirements X and Y, but not Z.
[Yigang] Okay, will consider to revise this section.

2. There needs to be a reference in the draft to the standard that defines SMS/MMS message formats. This is needed in order for us to see how it differs from email. Additionally, you need to specify whether these SMS/MMS spam report will be delivered via email. If not, this may not fall within MARF or IETF scope of work.
[Yigang] Do you need to see SMS format standards? Then, 3GPP/GSM MAP, CDMA SMDPP, SMPP, Parley, etc., too many. But spam report can be aligned with email.  

3. As per the MARF draft, there is a specific format to be followed when defining fields for the registry. You need to specify which abuse type it references, whether it is included once or multiple times, and whether it is historical. That format should be followed.
Additionally, ABNF would not hurt.
[Yigang] Once it is determined it falls within MARF scope, I can add the specific format.

4. The draft keeps referencing abuse detected by the message center.
However, the OMA SpamRep requirements talk about the end user reporting spam.
[Yigang] SMSC/MMSC and some gateway currently have different levels abuse detections. Therefore the reporting is bit different from the end user.

5. One of the basic principles underpinning MARF is that the data included in section 2 of the report, should be data that cannot be otherwise derived from the original message. Many of the fields that I see listed seem to be ones that will be included in the original message itself. There is no need to repeat them again in section 2 of the report.
[Yigang] To cover SMS/MMS, we should know whether the format is exactly same or not with email. Most of data cannot be included in original message if SMS report using email format.

Some field specific comments:
6. "Message ID" - in the email world, this is automatically generated in the headers of the MARF report itself.
[Yigang] In SMS world, this is generated by SMSC, while using different SMS protocol, the message ID will be in different parameters.
7. "Message Center ID" - in the email world the sender of the MARF report would be in the headers.
[Yigang] it is SMSC ID, it may or may not in SMS header depending on protocol.
8. "Message Type" - MARF is specific for email, IM reporting has its own formatting. You will need to create a registry or tightly define this. Also, is this information that would be included in the original message?
[Yigang] It depends how to include original message, just text or original headers, and depending on protocol.
9. Fields 4.4 - 4.15 - are these included in the original message? Is there a need to repeat them in part 2?
[Yigang] Depends.
10. Abuse types - is there a need for all of these? Prior discussion here seems to indicate that most of these are not necessary and some overlap. For example, message flooding and DOS are the same.
[Yigang] it is important when SMSC/MMSC determine abuse type.
11. Abuse detection methods - this is similar to the authentication results header we currently discussed regarding DKIM. There needs to be a document covering this.
[Yigang] Sure. I will be happy to do it.
12. Sections 4.18-4.23 - is any of this data included in the original message? Also, are any of these currently used?
[Yigang] depends.
13. Regarding feedback types - I would recommend looking at the possibility of defining a new feedback type called "sms" or "mobile"
and re-using as much as possible of existing MARF. For example, abuse keywords, etc. could be specified using URIs.
[Yigang] Sure. I can discuss further.

Yakov

---

From: "Alex Bobotek" <[email-masked]>
Date: Tue, 13 Jul 2010 09:25:38 -0700

In reference to #4 below, we should understand that OMA SpamRep was
written to cover reporting of SMS from an SMSC in addition to an SMS UA.
SpamRep is not limited to UA reporting.

Regards,

Alex

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Cai, Yigang (Yigang)
Sent: Tuesday, July 13, 2010 7:41 AM
To: Yakov Shafranovich; [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

Yakov,

Thanks,

Yigang

-----Original Message-----

As per emails to the list and the SpamRep documentation at the OMA, this
draft is trying to address the gap between MARF and the OMA SpamRep
requirements. One key point to note is that the MARF model is focused on
feedback loops between the ISP and the original email sender, while the
OMA SpamRep activity is focused on the end user and their network
provider. The analogous use case in the email space is a "report spam"
button for end users which creates a report for their ISP.
[Yigang] SMS/MMS abuse somehow should be covered either by OMA or MARF.
OMA SpamRep currently seems for a report from mobile device to server.
MARF is a report from a server to a server. 

1. Section 1.2 "requirements" - this seems to repeat the main MARF draft
requirements and seems to be different from the OMA requirements. There
is no need to repeat the MARF requirements verbatim, it may be
sufficient to reference them. The OMA requirements should be referenced,
and some of the OMA SpamRep requirements which cannot be solved with
MARF should be addressed explicitly here.
Ideally this section should clearly state that this draft seeks to
address OMA requirements X and Y, but not Z.
[Yigang] Okay, will consider to revise this section.

2. There needs to be a reference in the draft to the standard that
defines SMS/MMS message formats. This is needed in order for us to see
how it differs from email. Additionally, you need to specify whether
these SMS/MMS spam report will be delivered via email. If not, this may
not fall within MARF or IETF scope of work.
[Yigang] Do you need to see SMS format standards? Then, 3GPP/GSM MAP,
CDMA SMDPP, SMPP, Parley, etc., too many. But spam report can be aligned
with email.  

3. As per the MARF draft, there is a specific format to be followed when
defining fields for the registry. You need to specify which abuse type
it references, whether it is included once or multiple times, and
whether it is historical. That format should be followed.
Additionally, ABNF would not hurt.
[Yigang] Once it is determined it falls within MARF scope, I can add the
specific format.

4. The draft keeps referencing abuse detected by the message center.
However, the OMA SpamRep requirements talk about the end user reporting
spam.
[Yigang] SMSC/MMSC and some gateway currently have different levels
abuse detections. Therefore the reporting is bit different from the end
user.

5. One of the basic principles underpinning MARF is that the data
included in section 2 of the report, should be data that cannot be
otherwise derived from the original message. Many of the fields that I
see listed seem to be ones that will be included in the original message
itself. There is no need to repeat them again in section 2 of the
report.
[Yigang] To cover SMS/MMS, we should know whether the format is exactly
same or not with email. Most of data cannot be included in original
message if SMS report using email format.

Some field specific comments:
6. "Message ID" - in the email world, this is automatically generated in
the headers of the MARF report itself.
[Yigang] In SMS world, this is generated by SMSC, while using different
SMS protocol, the message ID will be in different parameters.
7. "Message Center ID" - in the email world the sender of the MARF
report would be in the headers.
[Yigang] it is SMSC ID, it may or may not in SMS header depending on
protocol.
8. "Message Type" - MARF is specific for email, IM reporting has its own
formatting. You will need to create a registry or tightly define this.
Also, is this information that would be included in the original
message?
[Yigang] It depends how to include original message, just text or
original headers, and depending on protocol.
9. Fields 4.4 - 4.15 - are these included in the original message? Is
there a need to repeat them in part 2?
[Yigang] Depends.
10. Abuse types - is there a need for all of these? Prior discussion
here seems to indicate that most of these are not necessary and some
overlap. For example, message flooding and DOS are the same.
[Yigang] it is important when SMSC/MMSC determine abuse type.
11. Abuse detection methods - this is similar to the authentication
results header we currently discussed regarding DKIM. There needs to be
a document covering this.
[Yigang] Sure. I will be happy to do it.
12. Sections 4.18-4.23 - is any of this data included in the original
message? Also, are any of these currently used?
[Yigang] depends.
13. Regarding feedback types - I would recommend looking at the
possibility of defining a new feedback type called "sms" or "mobile"
and re-using as much as possible of existing MARF. For example, abuse
keywords, etc. could be specified using URIs.
[Yigang] Sure. I can discuss further.

Yakov

---

From: "Cai, Yigang (Yigang)" <[email-masked]>
Date: Fri, 16 Jul 2010 15:08:08 -0500

All,

Considering standards and practice already allow SMS to carry email and IM, it is better to have a converged report format with multiple transport methods supported.

Yigang

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Yakov Shafranovich
Sent: Sunday, July 11, 2010 2:16 PM
To: [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

1. Thanks.
2. As of now, the intention of MARF is to be exchanged via email only.
It is a "son of" RFC 3462 which is "multipart/report" for administrative messages regarding email. Of course, other transport methods could be defined in the future.

Yakov

On Sun, Jul 11, 2010 at 1:40 PM, Wenjun (Kevin) Zeng <[email-masked]> wrote:
> between the end user and their network provider, the actual 
> specification only specifies two logical entities, i.e., a SpamRep 
> server and a SpamRep client where the client could be UA, MTA, SMSC 
> etc.
>

---

From: "Cai, Yigang (Yigang)" <[email-masked]>
Date: Fri, 16 Jul 2010 15:43:56 -0500

Alex,

I concur with many of your points, please see my comments inserted below.

Thanks,

Yigang

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Alex Bobotek
Sent: Sunday, July 11, 2010 3:19 PM
To: [email-masked]; Yakov Shafranovich; [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00": XML vs Headers

I believe that a single standard for mobile reporting (OMA and/or IETF) should be our goal, and am very interested in seeing OMA and IETF develop standards that do not fragment use and implementation, whether by having one or both bodies produce applicable standard(s).
[Yigang] I agree with this goal.

SpamRep was motivated by a desire to provide reporting from more than end users:  to provide a standard format for mobile reporting.  Yes, it is important to have a mobile spec that can be incorporated into phones by device manufacturers otherwise uninvolved in anti-abuse.  However, the SMS portion was writer to specifically support the following (and hopefully other) classes of reporting:

*  Manual reporting from an SMS UA to a network node
*  Automated reporting from an SMS UA to a network node
*  Automated/manual reporting from an SS7 or SMPP network node such as an SMSC
*  Intercarrier abuse notifications
[Yigang] In current service providers networks, the gateway and/or the SMSC already implemented spam filtering capability. It is much easier for them to report message abuse automatically.

SMS abuse reporting presents many challenges that are less prominent in email, notably that there are inherently many gateways (protocol
conversions) in the delivery path, and very little backtrace information (as in email headers). 
[Yigang] In some networks, SMS spam could be 80-90% of total SMS received. Since SMS is able to carry emails and IMs, some SMS spams are sourced from email and some email spams are sourced from SMS. 

In drafting the SMS portion of SpamRep, I found supporting multiple reporter classes to be a challenge, but not an impossibility.  Separate specifications seem to be an unnecessary and disadvantageous approach. 
[Yigang] Fully agree with you. 

While contributing to OMA SpamRep, OMA contributors (myself included) been following ARF/MARF, even using it as a baseline for many initial SpamRep requirements and features.  Two key differences between SpamRep and MARF are:

* SMTP vs HTTP (at least typically) for transport
[Yigang] From SMSC/MMSC point of view, SMTP and HTTP make not much difference w.r.t. development and testing effort. 
* XML vs RFC 5322 (email) style headers for report and message metadata
[Yigang] I prefer we converge to one.

IMO, either transport-independence (costly in that transport features such as encryption and authentication may be leveraged) or adaptation to multiple transports (my preference) is desirable and entirely feasible.

The key difference is that SpamRep is XML-based while thdraft-cai-smsc-mmsc-reporting-marf-00 draft and MARF is header-based.
At the root of this is that email works this way, while many other communications systems use XML.  Does anyone have any suggestions on how to drive convergence? 
[Yigang] I prefer to drive them to convergence.  

Regards,

Alex

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Wenjun (Kevin) Zeng
Sent: Sunday, July 11, 2010 10:41 AM
To: 'Yakov Shafranovich'; [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

Hi Yakov and all,

1. Although OMA SpamRep was originally motivated by feedback loop between the end user and their network provider, the actual specification only specifies two logical entities, i.e., a SpamRep server and a SpamRep client where the client could be UA, MTA, SMSC etc.

2. OMA SpamRep delivers Spam Reports through HTTP protocol. Is MARF restricted to email delivery of report only?

Thanks.

-Kevin

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Yakov Shafranovich
Sent: Sunday, July 11, 2010 10:53 AM
To: [email-masked]
Subject: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

Yakov

---

From: "Cai, Yigang (Yigang)" <[email-masked]>
Date: Fri, 16 Jul 2010 15:52:04 -0500

Yakov,

Some SMS spam is source of email spam considering SMS/email/IM can be converted either ways. It is a good the system receives abuse reports from multiple protocols and help to facilitate anti-spam policy in a centralized place.

Yigang

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Yakov Shafranovich
Sent: Sunday, July 11, 2010 11:10 PM
To: Alex Bobotek
Cc: [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00": XML vs Headers

On Mon, Jul 12, 2010 at 12:08 AM, Alex Bobotek <[email-masked]> wrote:
> OMA SMS SpamRep attributes were largely extracted from GSM MAP and 
> SMPP protocol fields.  But it is recognized that many attributes are not always available.  It is PREFERRED that accurate corpus be attached, but this isn't always possible.  In OMA SMS SpamRep there is metadata (i.e., DecodedUDIndicator and UDHAttached) that indicates whether or not, for example, the attached SMS corpus contains verbatim UD and UDH.  Another important metadata field (sometimes discernable from the message corpus) is SpamRep's 'ReportingNode' {SMS_Router, IP-SM-GW, SMS Router, IP-SM-GW, SMS-SC, Mobile_Device, Subscriber, Email_Gateway, Unknown, Other} attribute.
>
>> I believe that a single standard for mobile reporting (OMA and/or 
>> IETF) should be our goal, and am very interested in seeing OMA and 
>> IETF develop standards that do not fragment use and implementation, 
>> whether by having one or both bodies produce applicable standard(s).
>>
>> end users:  to provide a standard format for mobile reporting.  Yes, 
>> it is important to have a mobile spec that can be incorporated into 
>> phones by device manufacturers otherwise uninvolved in anti-abuse.  
>> However, the SMS portion was writer to specifically support the 
>> following (and hopefully other) classes of reporting:
>>
>> SMS abuse reporting presents many challenges that are less prominent 
>> in email, notably that there are inherently many gateways (protocol
>> reporter classes to be a challenge, but not an impossibility.  
>> Separate specifications seem to be an unnecessary and disadvantageous approach.
>>
>> SpamRep requirements and features.  Two key differences between 
>> SpamRep and MARF are:
>>
>> such as encryption and authentication may be leveraged) or adaptation 
>> to multiple transports (my preference) is desirable and entirely feasible.
>>
>> communications systems use XML.  Does anyone have any suggestions on 
>> how to drive convergence?
>>
>>
>>
>> Regards,
>>
>> Alex
>>
>>
>>
>> their network provider. The analogous use case in the email space is 
>> a "report spam" button for end users which creates a report for their 
>> ISP.
>>
>> verbatim, it may be sufficient to reference them. The OMA 
>> requirements should be referenced, and some of the OMA SpamRep 
>> requirements which cannot be solved with MARF should be addressed explicitly here.
>> defines SMS/MMS message formats. This is needed in order for us to 
>> see how it differs from email. Additionally, you need to specify 
>> whether these SMS/MMS spam report will be delivered via email. If 
>> not, this may not fall within MARF or IETF scope of work.
>>
>> when defining fields for the registry. You need to specify which 
>> abuse type it references, whether it is included once or multiple 
>> times, and whether it is historical. That format should be followed.
>> otherwise derived from the original message. Many of the fields that 
>> I see listed seem to be ones that will be included in the original 
>> this. Also, is this information that would be included in the 
>> original message?

---

From: "Cai, Yigang (Yigang)" <[email-masked]>
Date: Fri, 16 Jul 2010 16:01:28 -0500

Alex and All,

I agree with most of your points.

In addition, I have following comments:
(1) There are multiple SMS protocols even in a single network, such as 3GPP MAP, SMPP, CDMA SMDPP, SIP, Parley, etc. It is challenge to retrieve common fields from these protocols.
(2) SMSC/MMSC currently already developed some useful criteria to indicate spams, and which may not be a field in the protocol. But it is worth to include these fields optionally in abuse reporting. 
(3) The reporting may or may not contain the copy of original messages (email is easier to be attached). Most time, if abuse parameters/headers have sufficient indication, there is no need to include message body. SMSC/MMSC/email server can include part of message body which is considered as spam - current SMSC/MMSC already support the identification of spam phrase.  

Thanks,

Yigang

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Alex Bobotek
Sent: Sunday, July 11, 2010 11:09 PM
To: Yakov Shafranovich
Cc: [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00": XML vs Headers

Yakov,

OMA SMS SpamRep attributes were largely extracted from GSM MAP and SMPP protocol fields.  But it is recognized that many attributes are not always available.  It is PREFERRED that accurate corpus be attached, but this isn't always possible.  In OMA SMS SpamRep there is metadata (i.e., DecodedUDIndicator and UDHAttached) that indicates whether or not, for example, the attached SMS corpus contains verbatim UD and UDH.  Another important metadata field (sometimes discernable from the message corpus) is SpamRep's 'ReportingNode' {SMS_Router, IP-SM-GW, SMS Router, IP-SM-GW, SMS-SC, Mobile_Device, Subscriber, Email_Gateway, Unknown, Other} attribute.

Regards,

Alex

-----Original Message-----
From: Yakov Shafranovich [mailto:[email-masked]]
Sent: Sunday, July 11, 2010 4:34 PM
To: Alex Bobotek
Cc: [email-masked]; [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00": XML vs Headers

One of the main issues with the draft as it is, is the basic principle of MARF as it was designed - section 2, the machine readable section, should not contain any data that can be derived from the message itself. Because we don't know what standard you are using for defining SMS / MMS formats, I can't figure out whether the various message fields are already included in the original message.

Yakov

---

From: "Cai, Yigang (Yigang)" <[email-masked]>
Date: Fri, 16 Jul 2010 16:08:27 -0500

>From mobile UA, your comment is true. But from message center, it could be different, and I see there is a need of convergence.

Yigang

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Alex Bobotek
Sent: Monday, July 12, 2010 1:45 AM
To: John Levine; [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

I'm in agreement with John that MARF/Email and SpamRep/SMS are so far apart that it would be difficult to force them to converge.  Feedback and abuse management from a mobile SMS user agent has requirements for which HTTP (or any request/response protocol) is more advantageous than SMTP.  This is IMO true for any UA-based feedback, but especially so in the mobile case.  

SpamRep is a request/response transaction protocol in which some processing (with potential side effects) may be required prior to an application-layer response.  As written, it is not completely compatible with store & forward SMTP relay.  It would be awkward to mail back an asynchronous response to a device that has limited Internet connectivity, and some sort of extension to SMTP would likely be necessary for handling a MIME response.  Lemonade-like optimizations that prevent over-the-air transmission of messages already in possession of the receiving system are also needed.  SMTP is not an ideal transport for SpamRep.  

Emailing back protocol responses to cellular mobile devices (even if not over port 25, and especially if not immediate) isn't an attractive architecture.  A substantial subset of SpamRep could, in theory, operate over a message transport protocol such as SMTP.  

Note that SpamRep does support Email abuse reporting over HTTP.  Perhaps any overlap in this area should be discussed.

Regards,

Alex

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of John Levine
Sent: Sunday, July 11, 2010 10:22 PM
To: [email-masked]
Subject: Re: [marf] Review of "draft-cai-smsc-mmsc-reporting-marf-00"

>restricted to email delivery of report only?

MARF defines a MIME type.  To date, it's always been sent as the body of an email message, but HTTP also transmits MIME objects.

For that matter, since SpamRep is also a MIME type, you can send those, at least the ones that don't ask for an immediate response, by e-mail.

Given how little SMTP and SMS have in common, I'd be more inclined to keep them separate and encourage places that can accept both to accept them both by mail and HTTP.

R's,
John