Subject: [domainrep] Tentative requirements list

From: Alessandro Vesely <[email-masked]>
Date: Fri, 27 Aug 2010 12:19:00 +0200

I think it may help understanding if we spell what we mean by a Domain 
Reputation System (DRS).  One way to describe a conceptual model is to 
define a list of features, so that we can then poll each one.  Here's 
a tentative list, please add/change as appropriate.

*Small query*
Paramount simple properties or a domain reputation can be retrieved by 
short (UDP) DNS queries.

*Large query*
Complete raw data about a domain can be retrieved via HTTP in the form 
of an XML scheme.

*Taxonomy*
Each domain's activity is described by a code (to be determined). 
Possible uses of the code include estimating the likelihood of 
phishing, transactional mail, ad campaigns for 3rd parties, content by 
anonymous/unidentifiable users, liability for abusive behavior. (More?)

*Application specific quality*
A domain reputation is affected by its behavior.  Email, web, chat, 
and other apps (which ones?) contribute separate scores.

*Domains are ADMDs*
When we talk of /domain/ reputation we mean the ADMD, not the specific 
servers.  (While DKIM exhibits a "d=" tag that perfectly suits this 
purpose, I don't see the equivalent in other apps or SPF.)

*Private implementation*
Domain reputation data is collected and maintained by private 
organizations using their own "secret sauce".  Clients may choose one 
or more DRS, based on their preferences.

*Web of trust*
Domain reputation data are collected by a number of participating 
servers.  The participating domains are linked to one another in a 
directed graph with weights, thereby defining a web of trust. 
Maintenance of data is largely automatic, based on incident reports, 
abuse reports, and the like.

*MARF interface*
The trusted entity that provides reputation data for a given domain is 
among the targets of forwarded abuse reports.  DRS are a well-defined 
implementation of MARF, in the sense that implementation-specific 
fields like Reported-URI and Reported-Domain bear a well defined 
meaning, homogeneous across all DRS servers.

*IODEF interface*
Incident reports received by network providers' CSIRTs should affect a 
domain's reputation too, in case they can be unambiguously attributed 
to a domain.

*DNSBL interface*
Should these contribute directly to domains' reputation?  As for 
(some) incident reports, this would require a standard rule to get the 
domain responsible for a given IP.

---

From: "J.D. Falk" <[email-masked]>
Date: Fri, 27 Aug 2010 14:02:52 -0700

(as a participant)

On Aug 27, 2010, at 3:19 AM, Alessandro Vesely wrote:

> I think it may help understanding if we spell what we mean by a Domain Reputation System (DRS).  One way to describe a conceptual model is to define a list of features, so that we can then poll each one.  Here's a tentative list, please add/change as appropriate.
> 
> *Small query*
> Paramount simple properties or a domain reputation can be retrieved by short (UDP) DNS queries.

+1

> *Large query*
> Complete raw data about a domain can be retrieved via HTTP in the form of an XML scheme.

Probably, but I'm not entirely convinced yet.

> *Taxonomy*
> Each domain's activity is described by a code (to be determined). Possible uses of the code include estimating the likelihood of phishing, transactional mail, ad campaigns for 3rd parties, content by anonymous/unidentifiable users, liability for abusive behavior. (More?)

I agree with the possible uses, but I'm still concerned that we could argue about that list forever and never reach consensus -- and even if we did reach consensus, we won't have predicted every possible use case.  Better, I think, to let the Reputation Providers and Reputation Clients work that out later.

> A domain reputation is affected by its behavior.  Email, web, chat, and other apps (which ones?) contribute separate scores.

+1

> *Domains are ADMDs*
> When we talk of /domain/ reputation we mean the ADMD, not the specific servers.  (While DKIM exhibits a "d=" tag that perfectly suits this purpose, I don't see the equivalent in other apps or SPF.)

Probably +1, but I'm open to arguments to expand it.

> *Private implementation*
> Domain reputation data is collected and maintained by private organizations using their own "secret sauce".  Clients may choose one or more DRS, based on their preferences.

+1

> *Web of trust*
> Domain reputation data are collected by a number of participating servers.  The participating domains are linked to one another in a directed graph with weights, thereby defining a web of trust. Maintenance of data is largely automatic, based on incident reports, abuse reports, and the like.

-1 -- this is one possible implementation, but I wouldn't want to exclude others.

> *MARF interface*
> The trusted entity that provides reputation data for a given domain is among the targets of forwarded abuse reports.  DRS are a well-defined implementation of MARF, in the sense that implementation-specific fields like Reported-URI and Reported-Domain bear a well defined meaning, homogeneous across all DRS servers.
> 
> *IODEF interface*
> Incident reports received by network providers' CSIRTs should affect a domain's reputation too, in case they can be unambiguously attributed to a domain.
> 
> *DNSBL interface*
> Should these contribute directly to domains' reputation?  As for (some) incident reports, this would require a standard rule to get the domain responsible for a given IP.

-3 -- these are all possible implementations, and I know the MARF and DNSBL options are used today, but it should be left up to each individual Reputation Provider.

---

From: "J.D. Falk" <[email-masked]>
Date: Fri, 27 Aug 2010 14:03:20 -0700

Thanks for trying to codify these, BTW.  I'm looking forward to other replies.

---

From: Douglas Otis <[email-masked]>
Date: Fri, 27 Aug 2010 18:53:54 -0700

On 8/27/10 3:19 AM, Alessandro Vesely wrote:
>  I think it may help understanding if we spell what we mean by a
>  Domain Reputation System (DRS).  One way to describe a conceptual
>  model is to define a list of features, so that we can then poll each
>  one.  Here's a tentative list, please add/change as appropriate.
>
>  *Small query* Paramount simple properties or a domain reputation can
>  be retrieved by short (UDP) DNS queries.

To handle domains lacking path components, initial queries still need 
their depth constrained as a defense against DDoS attacks.  In other 
words, the query scheme should establish registrar's domains, and then 
only allow initial queries one level below these.  Registrar domains can 
be safely cached over long periods, and can be uniquely identified by 
having a specific category.

It would be good to standardize on a method for restricting results to 
clients subscribed to a service. Such restrictions help constrain 
resource overheads, and service related exposures.  Accessing the 
service through an ISP's resolver complicates this issue, however ISPs 
will likely want to avoid the level of DNS traffic that indirect access 
might generate.

Rather than attempting to subjectively describe domains as being "big" 
or "important", a list of the ongoing malicious activity instead offers 
the requester of the information a sound basis for their response to 
currently active threats, such as those defined in RFC5070.

1.  admin, Administrative privileges were attempted.
2.  dos.  A denial of service was attempted.
3.  file.  An action that impacts the integrity of a file or
     database was attempted.
4.  info-leak.  An attempt was made to exfiltrate information.
5.  misconfiguration.  An attempt was made to exploit a mis-
     configuration in a system.
6.  policy.  Activity violating site's policy was attempted.
7.  recon.  Reconnaissance activity was attempted.
8.  social-engineering.  A social engineering attack was
     attempted.
9.  user.  User privileges were attempted.
10. unknown.  The classification of this activity is unknown.
11. ext-value.  An escape value used to extend this attribute.
     See Section 5.1.

The requester, perhaps with help from other organizations or services, 
need to weigh currently reported threats against potential disruptions 
in commerce that the requester has with the specific domain.  This would 
be something that the requester should know far better than those 
detecting threats or abusive activity.  The requester of the information 
should already know in some manner with whom and to what level of 
commerce they or their customers have with a particular domain. After 
all, a decision to block is well beyond what threat investigators should 
make. The "secret sauce" should never be a secret to the requester of 
the information.  The requester needs concise and reliable information 
for the decisions they need to make.

Categories can be identified as an ENUM field (N) and incident counts 
(IC), which could be returned as an RR set of IPv4 addresses. Different 
RR sets can be placed within different sub-domains related to the 
specific service monitored, as John Levine had suggested.

A linear incident count has a tendency of exposing detection methods.  
Instead, incident counts should be expressed as a natural log value.  
This exponent matches well with detections seen as domains become larger 
and have more diverse exchanges.  As such, the incident count, (ICa or 
ICb), must never exceed 2.29 x 10^27 for the measured duration, or 
e^63.  This also allows Na or Nb to represent one of 31 different threat 
categories.

Additional threat categories can be specifically defined for the 
different services being measured, where the total categories being 
reported is limited to the number of addresses that can be returned 
within a DNS message.  When 13 IPv4 addresses can be returned within an 
RR set, then 26 different categories can be reported.

The value returned in each A record could be:

255 x 2^23 + Na x 2^17 + ICa x 2^11 + Nb x 2^6 + ICb

The fields would be within bits 31-23, 22-17, 16-11, 10-6, 5-0

(127+128<<23) + (Na << 17) + (ICa << 11) + (Nb << 6) + ICb

Without standardizing on the response returned by a service, there is 
little value in generically describing DNS as an amorphous solution 
later defined by various proprietary service providers.  What would be 
the point of writing such a document anyway?

>  *Large query* Complete raw data about a domain can be retrieved via
>  HTTP in the form of an XML scheme.

How this is handled really depends upon the nature and frequency of 
requests being made, and whether the information is being cached by the 
requester.   In some cases, this information can be a set of hashes, or 
elaborate XML structures.  This type of service using IODEF XML 
structures would help threat investigators to share their results and 
determine whether their own detection is being blocked.

>  *Taxonomy* Each domain's activity is described by a code (to be
>  determined). Possible uses of the code include estimating the
>  likelihood of phishing, transactional mail, ad campaigns for 3rd
>  parties, content by anonymous/unidentifiable users, liability for
>  abusive behavior. (More?)

Yes.  See above.

>  *Application specific quality* A domain reputation is affected by its
>  behavior.  Email, web, chat, and other apps (which ones?) contribute
>  separate scores.

Something like Threat Level Orange?  This places too much of the 
decision process on to the service collecting threat information.  In 
essence, a good/bad factor rating places defensive decisions onto the 
threat reporting service where they must also decide who is too big or 
too important to block.  The requester, should know this for themselves, 
or ask this from a different type of service.  This moves away from 
general and concise reporting of specific threats. The service should 
only need to share the overall activity monitored, and the duration that 
counts are held.

>  *Domains are ADMDs* When we talk of /domain/ reputation we mean the
>  ADMD, not the specific servers.  (While DKIM exhibits a "d=" tag that
>  perfectly suits this purpose, I don't see the equivalent in other
>  apps or SPF.)

It would seem the definition for domain depends upon the specific 
service being monitored for abuse.

>  *Private implementation* Domain reputation data is collected and
>  maintained by private organizations using their own "secret sauce".
>  Clients may choose one or more DRS, based on their preferences.

Such as how really big or important the domain is in the view of the 
service?  It seems the requester is better served by deciding this for 
themselves.  At least they would have no one else to blame when they 
make the wrong choice.

>  *Web of trust* Domain reputation data are collected by a number of
>  participating servers.  The participating domains are linked to one
>  another in a directed graph with weights, thereby defining a web of
>  trust. Maintenance of data is largely automatic, based on incident
>  reports, abuse reports, and the like.

This type of approach can be easily gamed by Bot-nets.  Large amounts of 
traffic can be exchanged amongst their 0wned systems to skew whatever 
metrics the agregator of the information uses.   Unless only 
AUTHENTICATED information is used to establish reputation, any domain, 
large or small, can be negatively impacted when assumed sources are used 
as a basis for metrics.  Perhaps companies like Dunn and Bradstreet are 
better able to offer dependable information than using a strange brew of 
someone's secret sauce.

>  *MARF interface* The trusted entity that provides reputation data for
>  a given domain is among the targets of forwarded abuse reports.  DRS
>  are a well-defined implementation of MARF, in the sense that
>  implementation-specific fields like Reported-URI and Reported-Domain
>  bear a well defined meaning, homogeneous across all DRS servers.

Worthless for establishing categories beyond that of spam.

>  *IODEF interface* Incident reports received by network providers'
>  CSIRTs should affect a domain's reputation too, in case they can be
>  unambiguously attributed to a domain.

Yes.  RFC5070 offers a good starting point.

>  *DNSBL interface* Should these contribute directly to domains'
>  reputation?  As for (some) incident reports, this would require a
>  standard rule to get the domain responsible for a given IP.

There should be a way of exchanging IODEF information between abuse 
reporting services.  This is already being done on a limited basis.

-Doug

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 28 Aug 2010 11:58:26 +0200

> Thanks for trying to codify these, BTW.  I'm looking forward to other replies.

Thank you for your support.  However, in order to elicit more 
responses, it may be useful if you (or some other chair candidate) 
will re-propose the call for a poll, as soon as the list seems final.

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 28 Aug 2010 12:27:46 +0200

Doug,
I really think we should settle what the topics are _before_ 
discussing their details.  Your contribution indicates that we should 
break the "small query" issue into what fields we want to appear in 
the response.  Beside "Taxonomy", I should have added

*Threat category and incident count*
The small query response includes recently reported incident counts, 
broken down into a fixed number of threat categories.

I also add this (see below)

*Emergency stop*
A reputation server shall signal to clients when a domain's server(s) 
seem to have been compromised.

On 28/Aug/10 03:53, Douglas Otis wrote:
> only allow initial queries one level below these. Registrar domains
> can be safely cached over long periods, and can be uniquely identified
> by having a specific category.

Do you mean that a resolver should block a query for

   some.deep.sub.sub.domain.of.dubious.example._rep.scorekeeper.example

unless it has already seen a query for

   dubious.example._rep.scorekeeper.example?

> will likely want to avoid the level of DNS traffic that indirect
> access might generate.

A decent mail or web server should have its own caching resolver. 
Possibly, client apps (mail readers, browsers) may eventually want to 
query a reputation service too.  I'm not sure this deserves much 
attention now.

> or "important", a list of the ongoing malicious activity instead
> offers the requester of the information a sound basis for their
> response to currently active threats, such as those defined in RFC5070.

Big or important is a relevant information, of course.

> in commerce that the requester has with the specific domain. This
> would be something that the requester should know far better than
> those detecting threats or abusive activity. The requester of the
> information should already know in some manner with whom and to what
> level of commerce they or their customers have with a particular
> domain. After all, a decision to block is well beyond what threat
> investigators should make. The "secret sauce" should never be a secret
> to the requester of the information. The requester needs concise and
> reliable information for the decisions they need to make.

If I've understood correctly, the question is how does 
scorekeeper.example tell me, given that I usually buy X from 
web.example, that web.example's server has recently been compromised? 
  Then, I should wait a while before giving them my CC number again. 
Is that what you mean?

Semantically, that's not the job of a reputation server.  Probably, 
reputation just tells me what's the likelihood that web.example can be 
compromised, and, in case it is, how well they can handle the 
incident.  At any rate, I add the "Emergency stop" entry above.

> [math omitted]

Agreed.  I'd also consider a time convolution, so that recent 
incidents weight much more than ancient ones.  In order to allow 
automatic updates, values should be maintained as real numbers.  These 
may be rounded or transformed so as to match the response format that 
will be determined.

> the point of writing such a document anyway?

That's a crucial point, IMHO.  Of course, reputation companies who 
have determined a recipe for a useful "secret sauce" and want to sell 
their service, would just do so.  Standardization will eventually 
happen later.  What are we doing, then?

The DAC experience has to be considered and digested before we 
proceed:  They have set up a certification service --the "big" and 
"important" facet of reputation--, have standardized it, and have 
failed, since I, for one, have not configured a list of trusted VBR 
certifiers, and don't know who would certify my domain.  What went 
wrong?  Did they contact the FDIC or whatever? (See the discussion 
between the two Johns in 
http://www.circleid.com/posts/fight_phishing_with_branding/)

IMHO, standardization becomes meaningful if it provides for mutual 
interaction among reputation trackers, so that clients can resolve 
queries, and results are consistent according to a client's trust.

> service being monitored for abuse.

Would it make sense to derive the domain from certificates' email 
addresses?

> This type of approach can be easily gamed by Bot-nets. Large amounts
> of traffic can be exchanged amongst their 0wned systems to skew
> whatever metrics the agregator of the information uses.

Yes, but if their contribution is correctly weighted according to the 
trust-path, only requesters who trust them will see it.

> Unless only AUTHENTICATED information is used to establish
> reputation, any domain, large or small, can be negatively impacted
> when assumed sources are used as a basis for metrics.

This is where DKIM becomes a useful anti-spam tool.

> Worthless for establishing categories beyond that of spam.

Yup, the idea is to track how well domains handle outgoing mail abuse.

> reporting services. This is already being done on a limited basis.

Someone familiar with those practices should say how to derive domain 
reputation from such data.  Is there anyone from INCH on this list?

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Sat, 28 Aug 2010 12:55:06 +0200

Hi, Alessandro,

> will re-propose the call for a poll, as soon as the list seems final.

working on the draft I have started analyzing all threads of the 
domainrep list. To prevent an endless stream of replies and forks of 
discussions, I suggest the following strategy: for each of the 
requirements in the list of Alessandro I start a thread with Subject 
<requirement> which provides a summary of what has been proposed, the 
pro's and con's and possibly a list of questions left open. Then each 
requirement can be discussed and at the end of the day the chair can 
start a call for a poll.

But first we need to finalize the list created by Alessandro (we can do 
this within this thread). As far as I can see most items has been 
covered; what I'm missing from the list is the 'Industry Taxonomy'. It 
has been discussed a number of times here and later on it was proposed 
to define a Threat Activity Taxonomy instead of an 'Industry Taxonomy'.  
For the moment I suggest to add the Industry Taxonomy; for each of the 
requirements we will discuss whether it really should be incorporated 
into the draft or whether it can be dropped.

Another one that I'd like to add is 'Scalability'. It will apply to both 
'small queries' as to 'large queries'.

BTW: I propose to change the name 'small query' and 'large query' to 
'small footprint reputation system' and 'large footprint reputation 
system', as a 'small query' can have a large amount of data being sent 
back to the requester.

/rolf

---

From: Douglas Otis <[email-masked]>
Date: Sat, 28 Aug 2010 18:11:38 -0700

On 8/28/10 3:27 AM, Alessandro Vesely wrote:
> discussing their details.
Details still help with imagining a view at 10,000 feet. :^)
>   Your contribution indicates that we should break the "small query" 
> issue into what fields we want to appear in the response.  Beside 
> "Taxonomy", I should have added
>
Be careful. Don't suggest threat monitoring services should make 
critical decisions of when to block a legitimate domain's service. There 
is always a compromised system issuing messages through any ISP's 
outbound server.  Since email is store and forward, which server has 
really been compromised?  With web services, there might be an iFrame 
injected in one of their customer's web-page's.  This leaves the basic 
question as to what represents an emergency?

The requester of threat information always needs to weigh threat 
incidents against their own relationship with the particular domain, and 
not expect blocking decisions will always be made for them by a 
monitoring service.  The requester should assess the relative importance 
of the sending domain, as balanced against their specific security 
needs. This is not a decision a monitoring service can make well, since 
any such decision would often be based upon wild speculation.

When an email provider serving the Netherlands is emitting malware spam, 
a requester located in Japan will likely apply  much lower blocking 
thresholds for this domain than a requester located in the Netherlands, 
based simply upon current levels of legitimate commerce.  The 
appropriate threshold also needs to reflect the security needs of the 
requester, such as a petrochemical plant will differ significantly from 
that of a flower shop.

In other words, a threat monitoring service should not be expected to 
know which domains are important to a requester.  Such a model will not 
work well at an Internet scale.  As such, threat service information 
should be combined with information gathered or obtained by the 
requester measuring their current level of commerce with the specific 
domain.  The requester, and not a threat monitoring service, will better 
understand the relative importance and the level of disruption a 
blocking decision might create, and only the requester can weigh this 
disruption against their security needs.

>   some.deep.sub.sub.domain.of.dubious.example._rep.scorekeeper.example
No. The defined query protocol used by the requester needs to allow a 
monitoring service a means to have cached results be applied against an 
entire branch of the name tree to squelch a common practice of using 
wild-cards with randomly generated sub-domains.  The monitoring service 
also needs to be able to enforce these requirements, which might mean 
giving a "go-away" answer for a particular client.  As such, only direct 
queries to the service should be used by clients, which can be made 
through their own local recursive resolvers.

unknown.co.uk would be okay as an initial query that then signals that 
all sub-domains are also blocked.

Initial queries at random.unknown.co.uk would mean a monitoring service 
will be inundated with queries for an infinite number of random domains 
that are essentially free for the bad actor.
> query a reputation service too.  I'm not sure this deserves much 
Before discussing what is needed to protect web browsing, please review

http://code.google.com/apis/safebrowsing/developers_guide_v2.html

Too many web pages have been compromised, where blocking at the domain 
would prove highly disruptive.
> Big or important is a relevant information, of course.
Agreed, but no threat monitoring service should be expected to know what 
is big or important from the perspective of the requester.  Perhaps 
firewall products used by a requester might track their commerce with 
these domains, and bind this information with accounting and exchange 
rates.  When requester specific information is then combined with 
current threat information, the firewall product can then make more 
appropriate blocking or warnings decisions tailored specifically for the 
requester.
>  Then, I should wait a while before giving them my CC number again. Is 
> that what you mean?
No.  A monitoring service should not be making assumptions about the 
requester.

Out of perhaps 26 categories of threats, perhaps one or two might relate 
to a service used to complete credit card transactions. When there are 
only a few recent incidents, and the card holder may be willing to 
closely monitor their card's activity, and decide not to block the domain.

On the other hand, when the requester is a network for an airport 
control tower, and a domain showing some threat activity is not 
essential for their critical operations and the requester has noted no 
prior history, then they might correctly decide to block at any incident 
level, where they have set more appropriate thresholds for essential 
domains, such as faa.gov, norad.mil, aviationweather.gov, noaa.gov. etc.
> incident.  At any rate, I add the "Emergency stop" entry above.
On what basis would this type of decision be made?
> incidents weight much more than ancient ones.
Stop expecting a centralized monitoring system is able to make critical 
domain blocking decisions for any requester.  This is not a model that 
will work in today's threat landscape.

Instead, allow the requester to interpret changes in incident 
information themselves.  The requester needs to maintain their own 
domain information.  Once a domain has been determined to be directly 
controlled by criminals, the reporting category should ensure any 
incident count offers sufficient warning against the entire domain.  
Perhaps threat monitoring services might also report failures in 
responding to threat reports, or incident count removals based upon the 
administrative trust extended by the monitoring service when told a 
category has been resolved.  Each category can also represent different 
windows of time.
>   In order to allow automatic updates, values should be maintained as 
> real numbers.  These may be rounded or transformed so as to match the 
> response format that will be determined.
A sliding window based upon integer measurements made every 300 seconds 
works fairly well, with summations reported over a sliding window of say 
3,600 and 86,400 seconds.  Just as the human ear has exponential 
sensitivity allowing comprehension over a high dynamic range, so would 
exponential counts.  To help a requester understand a change without 
them keeping a history, the reporting scheme could include two different 
measurement windows, where ICa might be 1 hour updated every 5 minutes, 
and ICb might be 1 day updated every hour, for example.
> happen later.  What are we doing, then?
It seems some believe that the approach taken to report on IPv4 
addresses can be directly applied to that of a domain, where the term 
"secret sauce" might be an unwillingness to admit security must then be 
secondary to a concern of generating too many support calls.
> queries, and results are consistent according to a client's trust.
Our company supported the Domain Assurance Council, but John Klensin is 
right on this issue, while seeing the issues differently.  One, paying 
an organization to say something about your domain will not deter bad 
actors. Someone will always take their money and a large number of 
domains will be willing to join them in a race to the bottom.  Secondly, 
schemes attempting to "push" accountability onto "authorized" domains 
have and are destine to fail.  The authorization scheme is easily gamed, 
since providers don't know what identity, if any, must correspond to an 
authorization.

This situation might change. The exchange of email from IPv6 addresses 
will eventually require cryptographic validation of SMTP clients.  An IP 
address seen by servers may not represent the IP address used by the 
client.  Having a more meaningful WHOIS tied to an IODEF monitoring 
system seems to represent the best path forward, provided there is the 
political will to hold providers responsible for monitoring their 
networks, and to identify the entities associated with domains engaged 
in commerce.
> Would it make sense to derive the domain from certificates' email 
Hopefully, domain reputation does not find itself limited to just email 
use.  There is an effort call keyassure that might be able to leverage 
DNSsec to allow use of public keys.  There is too much money involved 
with Internet crime to expect the cost of a certificate will do much to 
impair criminal activity.  Nor would I expect any service will be 
willing to single out individuals, or that such authenticated 
information will be generally available with respect to individual email 
addresses.
> trust-path, only requesters who trust them will see it.
Do you really think there is a way this can scale?  Look at how 
successful bad actors are at corrupting search results.  In this case, 
the trust path is normally within the same organization.  When some 
service attempts to distill the "strange brew" into something indicating 
domain X is bad, should domain X have any recourse?  When domain X 
prevails, will this then mean these services will need to relocate to 
countries where domains once again have no legal recourse?

> This is where DKIM becomes a useful anti-spam tool.
Disagree.  No one has ever claimed that DKIM is useful at preventing 
spam.  It is only useful at mitigating false positive detections of 
phishing attempts, and the white-listing of domains some consider "too 
big" to block.  In the latter case, this will actually increase the 
level of spam.

Putting in place a cryptographic mechanism to authenticate the SMTP 
client can make a difference in the level of spam by holding those who 
introduce the abuse accountable, and not pretending to hold those who 
may have done so accountable.  Wanting to believe DKIM really indicates 
who introduced the message overlooks that DKIM does not indicate where 
the message was directed, and that any message can be replayed.
> reputation from such data.  Is there anyone from INCH on this list?
Paul Ferguson might be a good person to ask.

-Doug

---

From: Paul Ferguson <[email-masked]>
Date: Sat, 28 Aug 2010 18:26:07 -0700

> Paul Ferguson might be a good person to ask.

Well, I don't claim to be "from INCH" -- since I didn't really
participate in the IETF WG -- but I did a bit of operational research
on this issue a couple of years ago when a similar topic was being
discussed on how CERTs (Computer Emergency Response Teams, not
Certificates) interact with one another in exchanging incident
information.

Part of what I do on a day-to-day basis is interact with lots of
people at CERTs globally, and of course, I think that IODEF could be
used in reporting security incidents and could certainly be adopted to
in a larger context.

The upshot of it is that IODEF, while defined, has not enjoyed
widespread usage in the CERT community, although there are a few
organizations which *do* use it -- the APWG and JP-CERT being two that
immediately leap to mind.

Pat Cain @ the APWG is the right guy to talk to on that (if anyone
needs his contact information, please ping me) and he was very active
in the INCH efforts w.r.t. IODEF.

With regards to deriving domain reputation metrics from that data, I
would think that would be a no-brainer, since the data is generated
based on "incidents" and methinks that would provide crucial
information regarding what reputation could be applied to a malicious
source/host/domain/whatever.

- ferg

-- 
"Fergie", a.k.a. Paul Ferguson
 Engineering Architecture for the Internet
 fergdawgster(at)gmail.com
 ferg's tech blog: http://fergdawg.blogspot.com/

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 29 Aug 2010 13:43:15 +0200

On 29/Aug/10 03:26, Paul Ferguson wrote:
> CERTs (Computer Emergency Response Teams, not Certificates)

Perhaps the term "CSIRT" has been coined to avoid such ambiguity?

> used in reporting security incidents and could certainly be adopted to

At this stage, it suffices to be able to estimate the extent of 
network providers involvement in the domain reputation effort.

As an email example, abusix.org sustain forwarding abuse reports to 
network providers (using yet another format :-O).  They don't take 
steps to ensure that recipients won't just drop such reports, though. 
  For our purposes, providers should feed (part of) incident data into 
the reputation system.  That should rise privacy concerns, especially 
in case they don't directly host the relevant "reputation server".

Presumably, different providers will want to engage at different 
degrees.  We should even provide for "criminal" network providers. 
But what is the "normal" level of involvement they may want to commit 
themselves to?

> immediately leap to mind.

Of course, life will be easier when formats will be standard.  Even if 
this WG-wannabe aims at standardization, I don't think the format if 
of foremost importance for the time being.  In that respect, this 
thread's subject is somewhat of a misnomer.

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 29 Aug 2010 15:49:29 +0200

> Details still help with imagining a view at 10,000 feet. :^)

Fine.

> Stop expecting a centralized monitoring system is able to make
> critical domain blocking decisions for any requester. This is not a
> model that will work in today's threat landscape.

Agreed.  Incidents may generally indicate a domain's quality, though.

>> The DAC experience has to be considered [...]
>> (See the discussion between the two Johns in
> Our company supported the Domain Assurance Council, but John Klensin
> is right on this issue, while seeing the issues differently.

One point is that it is not worth to insert lots of 
difficult-to-determine configuration details for only bettering a tiny 
part of traffic.

> One, paying an organization to say something about your domain will
> not deter bad actors. Someone will always take their money and a
> large number of domains will be willing to join them in a race to
> the bottom.

IMHO, the service should be provided for free.

> Secondly, schemes attempting to "push" accountability onto
> "authorized" domains have and are destine to fail. The authorization
> scheme is easily gamed, since providers don't know what identity, if
> any, must correspond to an authorization.

For email, a DKIM-signed abusive message inside a DKIM-signed abuse 
report can be assimilated to an incident, unless the original sender 
contests.  Yet, incidents should not directly affect a domain's 
reputation.  It is the /domain response/ to the incident that matters.

>> if [each actors'] contribution is correctly weighted according to
>> the trust-path, only requesters who trust them will see it.
> Do you really think there is a way this can scale?

It should scale just like IP allocations and domain delegations do.

> When some service attempts to distill the "strange brew" into
> something indicating domain X is bad, should domain X have any
> recourse? When domain X prevails, will this then mean these
> services will need to relocate to countries where domains once
> again have no legal recourse?

Possibly not.  I haven't worked out a model, but the idea is that 
trust can be chained.  For a rough example, assume A trusts B and B 
trusts C, with weights:

    A --0.8--> B --0.8--> C

Thus A trusts C with weight 0.64, in a multiplicative model.  Now, say 
C does not handle an incident that A reported to B.  Because of that, 
B may lower its trust toward C to 0.4.  If C prevails and B doesn't 
adjust its score, A may lower its trust toward B, e.g. again to 0.4. 
Thus, the final scores are either

    A --0.8--> B --0.4--> C
or
    A --0.4--> B --0.8--> C

In both cases, the score A-->C drops to 0.32.  (Other circumstances 
may cause scores to raise, there may be multiple paths, and further 
intricacies to account for.)

The above is just an oversimplified example that I hope renders the 
idea.  Node B may be bad because of any reason, including legal 
coercion.  However, B's friends may continue to trust it, and it may 
still work for them; a "global walled garden" does not have to be 
uniformly connected.

Initial trust should be granted to network providers, registries and 
similar entities.  Each domain should provide its server.  Recourses 
and contestations should seldom cause human interventions because 
false claims can easily be punished by lowering scores.  The 
difficulty is to enumerate all possible operations, and then elaborate 
a model that will track them correctly, allowing human intervention as 
appropriate.  A case of collective intelligence?

---

From: Douglas Otis <[email-masked]>
Date: Sun, 29 Aug 2010 11:46:05 -0700

On 8/29/10 6:49 AM, Alessandro Vesely wrote:
> > critical domain blocking decisions for any requester. This is not
> > a model that will work in today's threat landscape.
>

A centralized monitoring system will not have a basis in which to judge 
quality, except at toxic extremes.  An accurate metric for overall 
volume is needed to make reasonable quality assessments.  For example, 
water quality is reported in parts per million or billion.  Incidents 
expressed as quality by a monitoring service would be speculating about 
volume.

Such speculated volumes can be significantly influenced by bad actors.  
A requester or network provider, on the other hand, is able to 
accurately monitor their own volume metrics.  Incident monitoring offers 
better information when it is not polluted by speculated volumes and 
reported as bogus quality measurements.

Leave the decision of quality to network providers, and decisions of 
when to act to the requester.  Mixing known reported incidents with 
speculated volumes reduces the information quality.  If anything, report 
volume speculations as a separate category, to that of incident 
reporting, perhaps as category 0.

>  difficult-to-determine configuration details for only bettering a
>  tiny part of traffic.

Do you mean reporting the category of commerce?  If so, then I would 
agree the costs would not outweigh the benefit.  Bad actors could then 
easily have themselves appear as being in one of these "too important" 
or "too big" categories.

> > One, paying an organization to say something about your domain
> > will not deter bad actors. Someone will always take their money and
> > a large number of domains will be willing to join them in a race
> > to the bottom.
>

Do you mean that resource fees should offer adequate enforcement of 
related agreements?  But even that is not really free.  On the other 
hand, a monitoring service as a stop-gap effort will deal with DDoS 
attack and lawsuits, whose defense requires adequate funding.  Or said 
another way, you don't get what you don't pay for.

> > "authorized" domains have and are destine to fail. The
> > authorization scheme is easily gamed, since providers don't know
> > what identity, if any, must correspond to an authorization.
>

Disagree.  There is nothing preventing bad actors from inundating the 
Internet with any domain's DKIM messages.  A bad actor could obtain a 
Yahoo account, send themselves their spammy messages, and then replay 
them throughout the Internet.  Should a domain abused by a monitoring 
service who wrongly holds a DKIM domain responsible for the actions of 
an unknown SMTP client really be expected to contest each injustice?  
What happens when the DKIM domain responds with a request to stop 
sending the abusive reports!  Would this then allow the monitoring 
service to report the DKIM domain as unresponsive and a bad domain?

> >> if [each actors'] contribution is correctly weighted according
> >> to the trust-path, only requesters who trust them will see it.

Bad actors can indicate they trust their hundreds of thousands of 0wned 
systems.  They can say bad things about your domain and replicate what 
has been said about the others, including their own which they will have 
managed to skew.  They then send you a friendly message indicating they 
can improve your domains rating, at a moderate fee.

Scaling trust on a free basis is something bot-nets are good at gaming.  
Time to put away the tie-dyed shirt, the era of free-trust is over. :^)

>  Thus A trusts C with weight 0.64, in a multiplicative model.  Now,
>  say C does not handle an incident that A reported to B.  Because of
>  that, B may lower its trust toward C to 0.4.  If C prevails and B
>  doesn't adjust its score, A may lower its trust toward B, e.g. again
>  to 0.4. Thus, the final scores are either
>
>  A --0.8--> B --0.4--> C or A --0.4--> B --0.8--> C
>
>  difficulty is to enumerate all possible operations, and then
>  elaborate a model that will track them correctly, allowing human
>  intervention as appropriate.  A case of collective intelligence?

Let me quote the late great Frank Zappa, "Dumb all over and a little 
ugly on the side."   When the basis for judging reputation is 
fundamentally flawed, does it really matter how many are willing to make 
the same mistake?

-Doug

http://www.youtube.com/watch?v=YGZ5isu23ow

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 30 Aug 2010 13:21:13 +0200

On 29/Aug/10 20:46, Douglas Otis wrote:
> agree the costs would not outweigh the benefit.

Including a huge list of trusted vouchers is hard, unless we introduce 
meta-vouchers (looks like pre-loaded X509 certifiers.)

> Bad actors could then easily have themselves appear as being in one
> of these "too important" or "too big" categories.

Not in the FDIC, say.  If all vouchers are carefully selected, none of 
them would vouch for unionbanknigeria.net.

> another way, you don't get what you don't pay for.

If it is my network provider, a national institution, or similar, 
resources are already paid for.  An private independent service will 
have difficulties in gaining credibility; however, if it manages to do 
that business, they may charge some amount for /querying/, not for 
vouching. (Spamhaus won't take money to unlist addresses.)

> an unknown SMTP client really be expected to contest each injustice?

Yahoo should fire that account, as a response.  Their policy for 
getting new accounts being what it is, they may have to do it several 
times.

For the mechanics of replaying, you're right: DKIM signatures /alone/ 
do not attest the legitimacy of a message.

> service to report the DKIM domain as unresponsive and a bad domain?

Reports should never be abusive, since they should only be shared by 
trusting parties.  However, they can be sent by error, or in response 
to an attack.

> Bad actors can indicate they trust their hundreds of thousands of
> 0wned systems. They can say bad things about your domain and replicate
> what has been said about the others, including their own which they
> will have managed to skew. They then send you a friendly message
> indicating they can improve your domains rating, at a moderate fee.

Yeah, they also send 419 opportunities :-)

> Scaling trust on a free basis is something bot-nets are good at
> gaming. Time to put away the tie-dyed shirt, the era of free-trust is
> over. :^)

Despite banks often abusing that term, "trust" cannot really be 
bought, nor sold.

>> [snipped vision of collective intelligence]
>
> fundamentally flawed, does it really matter how many are willing to
> make the same mistake?

Yes, it does! Frank's lyrics just implied it :-)  Besides, I wouldn't 
despair of getting reputation judgments right.

Even if we decide that we won't implement a web of trust, we should 
strive to avoid standardizing models that preclude it, if at all possible.

> http://www.youtube.com/watch?v=YGZ5isu23ow

Mighty grand, thanks!

---

From: Nathaniel Borenstein <[email-masked]>
Date: Mon, 30 Aug 2010 07:57:48 -0400

Alessandro's "small and larger queries" -- which I like -- seem to point towards a deeper aspect of the problem, and towards a few simple things we can do to make the resulting protocol much more generally useful.  What I'm about to suggest doesn't affect most of what we've discussed on this list, only the structure of the reputation information being passed around.

Most of us are on this mailing list because we want to use reputations to attack email-related problems.  We're inevitably going to worry primarily about the aspects of reputation that apply to email.  We probably all agree on the need to itemize threat types; we haven't (yet) all agreed on several other likely things such as organization types, but they're obviously valid topics to discuss.  Still, most of us want to exclude from our scope the management of reputations for activities such as kiddie porn, music piracy, aiding  terrorists, swindling consumers, etc. etc.  But it's hard to deny that strong needs for reputation exchange will emerge in all sorts of domains beyond email.  We should at least consider trying to get the basic idea right in the first place, to make the system potentially reusable for such purposes.

I'd like to view Allesandro's large and small queries in the context of long-term extensibility versus short term needs.  If we can create a good enough data model for reputation information, the small query (email system using DNS) could effectively express an interoperable subset of the large query (web-based systems doing arbitrarily complex stuff).  So, I find myself wondering: what is the fundamental unit of reputation exchange?

The simplest "reputon" I can think of -- though I fear I'll shortly learn just how oversimplistic it is -- looks like this strawman idea:

AGENT-A (identified with confidence level X) says that AGENT-B has PERCENTAGE likelihood (with confidence level Y) of ASSERTION.  (Some assertions will probably require encapsulating some arbitrary data as part of the reputon.)

More sophisticated information can be computed by matching multiple reputons and facts together.  The weight we give the above reputon might depend on  another reputon, assessing AGENT-A's value as a source of reputation information.  (Or we might base that assessment on our own past observations.)  

With the right definition of a reputon, we can easily devise syntax for using it in the small and large queries.  Each application using the mechanism will define its own values of ASSERTION.  (I'm guessing we need a two-level namespace here.)  I think we have the chance to produce a pretty useful piece of infrastructure here, at very little (if any) additional cost to us.  It also suggests a divide-and-conquer approach here -- one document could define a MIME type for reputons, another the overall architecture for reputation management, two more could define the small and large query types, and another the specific ASSERTIONs for email reputations.

Thoughts?  -- Nathaniel

---

From: Douglas Otis <[email-masked]>
Date: Mon, 30 Aug 2010 15:29:25 -0700

On 8/30/10 4:21 AM, Alessandro Vesely wrote:
>  Including a huge list of trusted vouchers is hard, unless we
>  introduce meta-vouchers (looks like pre-loaded X509 certifiers.)

There may be a confirmation process developed to support a keyassure 
public key mechanisms that allows government de-certification, and 
confirmation of a domains' physical locations.

>  Not in the FDIC, say.  If all vouchers are carefully selected, none
>  of them would vouch for unionbanknigeria.net.

Are you suggesting EV certificates would not be good enough for 
exchanging email?  Will governments then need to confirm the new 
certificate type, and who will pay for the confirmation?

> > Do you mean that resource fees should offer adequate enforcement
> > of related agreements? But even that is not really free. On the
> > other hand, a monitoring service as a stop-gap effort will deal
> > with DDoS attack and lawsuits, whose defense requires adequate
> > funding. Or said another way, you don't get what you don't pay
> > for.
>
>  have difficulties in gaining credibility; however, if it manages to
>  do that business, they may charge some amount for /querying/, not for

Okay, we agree service subscriptions might be used.  This then requires 
a means to enforce query behavior and subscription payments, or more 
generally, a mechanism is needed to exclude non-subscribers or bad-actors.

> >> For email, a DKIM-signed abusive message inside a DKIM-signed
> >> abuse report can be assimilated to an incident, unless the
> >> original sender contests. Yet, incidents should not directly
> >> affect a domain's reputation. It is the /domain response/ to the
> >> incident that matters.
> >
> > Disagree. There is nothing preventing bad actors from inundating
> > the Internet with any domain's DKIM messages. A bad actor could
> > obtain a Yahoo account, send themselves their spammy messages, and
> > then replay them throughout the Internet. Should a domain abused by
> > a monitoring service who wrongly holds a DKIM domain responsible
> > for the actions of an unknown SMTP client really be expected to
> > contest each injustice?
>

So what if they do?  Even accounts promptly disabled will not halt 
distributions of replayed messages, nor will rate limiting user accounts 
offer protection from replay abuse.  A feature of DKIM is that a message 
can be redirected anywhere. DKIM domains can not be expected to contest 
each injustice of being wrongly accused of having assigned a message 
destination.

>  do not attest the legitimacy of a message.

A DKIM domain does not identify who assigned the destination for an 
unsolicited message, especially when adopting a principle that 
assessment MUST allow domains a means to defend their reputation.  A 
bad-actor will not need to be a user of the domain to replay messages.  
The SMTP client domain, and not the DKIM domain, MUST be used to 
establish reputations based upon receipt of unsolicited messages.

>  trusting parties.  However, they can be sent by error, or in response

Reports should always be directed to a domain verified as having issued 
the message to the recipient.  DKIM does not offer such verification.  
It is not right to say since most messages are not replayed, it is safe 
to assume a message was not replayed when assessing reputations or when 
deciding where to send notifications.  You know what they say about 
making assumptions...

> > 0wned systems. They can say bad things about your domain and
> > replicate what has been said about the others, including their own
> > which they will have managed to skew. They then send you a friendly
> > message indicating they can improve your domains rating, at a
> > moderate fee.
>

It is also difficult to detect and remove all outbound messages offering 
drugs, software, nude pictures of hot celebrities, undelivered packages, 
politicians caught in the act, claiming someone sent a large amount of 
spam, or that someone failed to pay for a service.  Any such message 
might be replayed and used to entice victims into visiting malicious 
drive-by websites.  At the same time, one of their ten of thousands of 
0wned systems can also apply for new accounts, while granting access to 
pornography for those solving forwarded captchas.

> > gaming. Time to put away the tie-dyed shirt, the era of free-trust
> > is over. :^)
>

Certificates, vouching, and CA services are being bought and sold 
because people feel they can trust this information.  The results of a 
searching service are intended to indicate where people are referencing 
a specific topic.  Unfortunately,  bot-nets continuously demonstrate a 
skill at corrupting search results obtained from the community.  A large 
number of anything is progressively becoming less meaningful.

>  strive to avoid standardizing models that preclude it, if at all

No matter how reputation information is distributed, evidence supporting 
reputations MUST NOT be based upon any assumption of involvement.  
Holding an entity accountable for an unsolicited message means no 
assumptions regarding who assigned the destination can be allowed.  
Otherwise, bad-actors can quickly make such a system untrustworthy, 
disruptive, and wholly unfair.

-Doug

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Tue, 31 Aug 2010 00:36:57 +0200

Nathaniel,

the model you propose sounds attractive to me, but I'm not sure I 
understand all the details right now. A few questions:

On 08/30/2010 01:57 PM, Nathaniel Borenstein wrote:
> Alessandro's "small and larger queries"

(minor detail) I believe John was the first one to propose the split 
between small and large domains/queries.

> -- which I like -- seem to point towards a deeper aspect of the 
> problem, and towards a few simple things we can do to make the 
> resulting protocol much more generally useful.  What I'm about to 
> suggest doesn't affect most of what we've discussed on this list, only 
> the structure of the reputation information being passed around.
>
> Most of us are on this mailing list because we want to use reputations 
> to attack /email-related /problems.  We're inevitably going to worry 
> primarily about the aspects of reputation that apply to email.  We 
> probably all agree on the need to itemize threat types; we haven't 
> (yet) all agreed on several other likely things such as organization 
> types, but they're obviously valid topics to discuss.  Still, most of 
> us want to exclude from our scope the management of reputations for 
> activities such as kiddie porn, music piracy, aiding  terrorists, 
> swindling consumers, etc. etc.  But it's hard to deny that strong 
> needs for reputation exchange will emerge in all sorts of domains 
> beyond email.  We should at least consider trying to get the basic 
> idea right in the first place, to make the system potentially reusable 
> for such purposes.
>
> I'd like to view Allesandro's large and small queries in the context 
> of long-term extensibility versus short term needs.  If we can create 
> a good enough data model for reputation information, the small query 
> (email system using DNS) could effectively express an interoperable 
> subset of the large query (web-based systems doing arbitrarily complex 
> stuff).

+1

>  So, I find myself wondering: what is the fundamental unit of 
> reputation exchange?
>
> The simplest "reputon"

Very nice name!

> I can think of -- though I fear I'll shortly learn just how 
> oversimplistic it is -- looks like this strawman idea:
>
> AGENT-A (identified with confidence level X) says that AGENT-B has 
> PERCENTAGE likelihood (with confidence level Y) of ASSERTION.  (Some 
> assertions will probably require encapsulating some arbitrary data as 
> part of the reputon.)

Can you explain more about this 'encapsulating some arbitrary data', 
possibly provide a use case or an example?

> More sophisticated information can be computed by matching multiple 
> reputons and facts together.  The weight we give the above reputon 
> might depend on  another reputon, assessing AGENT-A's value as a 
> source of reputation information.  (Or we might base that assessment 
> on our own past observations.)

Probably our own past observations should be expressed in reputons, to 
fit nicely into this model?

> With the right definition of a reputon, we can easily devise syntax 
> for using it in the small and large queries.  Each application using 
> the mechanism will define its own values of ASSERTION.  (I'm guessing 
> we need a two-level namespace here.)

What two-level namespace do you have in mind? Can you give an example?

>  I think we have the chance to produce a pretty useful piece of 
> infrastructure here, at very little (if any) additional cost to us. 
>  It also suggests a divide-and-conquer approach here -- one document 
> could define a MIME type for reputons, another the overall 
> architecture for reputation management, two more could define the 
> small and large query types, and another the specific ASSERTIONs for 
> email reputations.
>
> Thoughts?  -- Nathaniel

It's an interesting model, but as I said, I'm not sure I understand all 
the details, yet.

/rolf

---

From: Scott Kitterman <[email-masked] >
Date: Mon, 30 Aug 2010 19:55:59 -0400

>confirmation of a domains' physical locations.
Unless methods for determining reputation are part of this effort,  this seems not relevant. 

>certificate type, and who will pay for the confirmation?

This is really up to the reputation provider to get right. I don't see anything to standardize.

>generally, a mechanism is needed to exclude non-subscribers or bad-actors.

Why is more mechanism needed for domain reputation than. IP reputation?  There are plenty of crap RBLs out there and good admins learn to avoid them.

>each injustice of being wrongly accused of having assigned a message 

They can and they will.  If you sign a message,  you take responsibility.

>establish reputations based upon receipt of unsolicited messages.

DKIM signing domain will be one input into reputation because signing a message is taking responsibility for it.

>deciding where to send notifications.  You know what they say about 

Yes. It's probably a bad idea to assume reputation providers will care if a signing domain tries to avoid responsibility for messages they've signed by claiming they were replayed. 

>pornography for those solving forwarded captchas.

To the extent entities do well at this, they will have good reputations.  To the extent they don't, they won't. 

>number of anything is progressively becoming less meaningful.

Low quality aggregation is not helpful. They key is for providers to do it well, but that's not very related to anything that can be standardized.

>disruptive, and wholly unfair.

It's up to reputation providers to work out how to do this and not a subject for standardization.

Scott K

---

From: Nathaniel Borenstein <[email-masked]>
Date: Tue, 31 Aug 2010 14:24:14 -0400

On Aug 30, 2010, at 6:36 PM, Rolf E. Sonneveld wrote:

> (minor detail) I believe John was the first one to propose the split between small and large domains/queries.

Oops, sorry.  If the Internet were a crime scene John would be in trouble, his prints are all over it.  :-)

> Can you explain more about this 'encapsulating some arbitrary data', possibly provide a use case or an example?

Sure.  The intent of my model -- which so far exists only in the fuzzy logic of my brain -- was to make the reputon the simplest, most generic expression of reputation data possible.  Then, the act of defining each ASSERTION would define the contents of the arbitrary MIME entity that accompanies the reputon.

What I see as the strength of this approach is that we don't have to argue so much about what kind of data gets included in the general case; additional data will be defined for each assertion

So, in the case of rating email sources, we might want to talk about a reputation for sending spam, and such a reputation might be more useful if accompanied by the date range for which spamming was evidenced.   In this case, the appended data would include those date ranges, iin a format specified as part of the definition of the assertion.  Or, for the assertion that a domain is sourcing email that contains viruses, the attached data might include a description of the type of virus.

Or not.  The point is that we won't have to argue about these details globally, but in specific contexts where it's easier to see if something is useful.

> Probably our own past observations should be expressed in reputons, to fit nicely into this model?

I think that's an implementation question.  If I had a good symbolic logic system, I might consider translating the reputons into its syntax instead.  But I think most uses will be far too simple to require such a system.   On the other hand, if this catches on, I can certainly imagine some logic engines operating on them directly.

>> With the right definition of a reputon, we can easily devise syntax for using it in the small and large queries.  Each application using the mechanism will define its own values of ASSERTION.  (I'm guessing we need a two-level namespace here.)
> 

I almost wrote "at least two-level" actually, because I'm increasingly thinking about three levels.  I was imagining letting anyone with an application define reputation assertions for that application, with domain names used to prevent naming conflicts.   And it might make sense to help those domains support multiple applications with more than a one-level name space, so I'd consider including a general category name like email, yielding something like:

	consumer-report.com/dentists/is-liberal-with-nitrous-oxide
or
	ietf.org/email/uses-punctuation-sloppily

In the latter case, the IETF would own the namespace, make its own decisions about the category names, and have a mechanism for publishing the definition of the assertion.  

In a nutshell:  I'm trying to decouple the definition of what you're asserting from the mechanism by which you assert it.  -- Nathaniel

---

From: Douglas Otis <[email-masked]>
Date: Tue, 31 Aug 2010 15:39:59 -0700

On 8/30/10 4:55 PM, Scott Kitterman wrote:
> "Douglas Otis"<[email-masked]>  wrote:
Scott,

Criminal activity authenticated by a certificate provides a fairly solid 
basis for authorities to then demand removal of the certificate along 
with the related domain. This function needs to be part of any public 
key scheme using DNS, where immediate removal would not be otherwise 
possible.  Most domain exploitations begin and end within a few hours.  
Such a removal list could also make reputation services unneeded.
> This is really up to the reputation provider to get right. I don't see anything to standardize.
So reputation erects a barrier high enough to get everyone paying for 
the added efforts?  First start by getting everyone to properly 
authenticate their domain.

Why not simply use TLS with a EV cert with SMTP clients and call it 
done?  Why would there be a need for reputations at that point, since 
promptly and proactively responding to criminal activity would be 
expected, would it not?
> Why is more mechanism needed for domain reputation than IP reputation?  There are plenty of crap RBLs out there and good admins learn to avoid them.
Sorry, this must not have been said clearly enough.  The service  needs 
a way to indicate "service is being refused", without causing some other 
problem, where refusal also needs to occur at the initial stages.
> They can and they will.  If you sign a message,  you take responsibility.
A domain applying DKIM can only take responsibility for the signed 
portion of the message.  DKIM does NOT affirm the destination or return 
path of the message, which means DKIM domains can not be held 
responsible for the sending of unsolicited email, since a DKIM signature 
does not affirm the signing domain's involvement in this abusive act.
> DKIM signing domain will be one input into reputation because signing a message is taking responsibility for it.
Signing a message using DKIM MUST NOT be considered "taking 
responsibility" for actions of subsequent MTAs. After all, use of 
subsequent MTAs permit destinations to be modified, and messages 
replayed.  These actions are well beyond the control of the signing 
domain.  Reasons for using DKIM is largely to guard against spoofing the 
source of a message.   In the course of doing my job, I might send 
samples of messages used to entice victims to my colleagues, who then 
take the needed precautions.  However, if such a message were to become 
replayed and widely distributed, who would be acting irresponsibly? The 
entity initially sending the message, or those granted access to a 
subsequent MTA that allows destinations to be modified, and then 
replayed to large numbers of unskilled recipients?

Large domains might rate-limit accounts to protect their systems from 
being associated with bulk unsolicited email.  A bad-actor could 
establish a new account and then send themselves messages of which they 
intent to replay well beyond reason.   Once the large domain discovers 
the abuse, they will be unable to stop the campaign.  Why should the 
large domain offering low cost accounts be held accountable for abuse 
they can not be expected to control?
> Yes. It's probably a bad idea to assume reputation providers will care if a signing domain tries to avoid responsibility for messages they've signed by claiming they were replayed.
Are you suggesting reputation services can ignore the domain of the MTA 
issuing abusive messages, whenever they find it easier to use 
questionable message content to make baseless assumptions about the 
source of a message?  Such a service could easily find themselves facing 
serious consequences for such negligent assumptions.

A bad-actor is able to violate any relationship normally seen between 
various message elements.   Having signed a message does not mean the 
domain intended the destination of the message where it was not wanted, 
nor does having authorized a server mean this domain originated the message.
> To the extent entities do well at this, they will have good reputations.  To the extent they don't, they won't.
Having written some of these filters, it is very difficult to block more 
than 97% of messages created by clever bad actors.  While the filter 
might block a high percentage of outbound abusive messages, those that 
slipped past will have also been vetted as difficult to detect and 
become good candidates for being replayed a million-fold.  No one can 
ever filter outbound messages well enough to protect their reputation 
whenever DKIM domains become a basis for acceptance and are held 
accountable for the uncontrolled destination of the message.  In other 
words, this scheme will fail.
> Low quality aggregation is not helpful. They key is for providers to do it well, but that's not very related to anything that can be standardized.
While I often use several different search engines, none of these 
services seem able to prevent results from being incredibly skewed.  
There is even an underground industry working at corrupting these services.
> It's up to reputation providers to work out how to do this and not a subject for standardization.
Why would it be wrong to suggest domains being held accountable need to 
be authenticated when counting incidents that might negatively affect a 
domain's ability to conduct commerce.  Are you suggesting there are no 
standards or methods to authenticate a domain?  Frankly, IPv6 will soon 
make most IP address related methods impractical, so we should be 
working earnestly at developing standards needed to authenticate SMTP 
client domains that can be held accountable for the entirety of a 
message issued publicly on port 25.

-Doug

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 31 Aug 2010 19:05:46 -0400

> Such a removal list could also make reputation services unneeded.

Which reinforces my point that it's not related to this effort.

> promptly and proactively responding to criminal activity would be

No.

> > Why is more mechanism needed for domain reputation than IP reputation? 
> > There are plenty of crap RBLs out there and good admins learn to avoid
> > them.
> 

No.  It actually doesn't mean that at all.

> they can not be expected to control?

So I guess we should just give up on DKIM since it's useless as an input for 
reputation for anything else?  In any case, this is off topic here, since 
assessing the reputation of domain is up to a reputation provider and not 
something we can standardize.

> > Yes. It's probably a bad idea to assume reputation providers will care if
> > a signing domain tries to avoid responsibility for messages they've
> > signed by claiming they were replayed.
> 

I'm suggesting that people will consider signing domain from DKIM as an input 
into their reputation calculations since that's been advertised as it's 
primary use, but, once again, that's up to reputation providers to figure out 
and not subject to standardization.

> words, this scheme will fail.

Then such reputations won't be usable in the real world and reputation 
providers will have to figure out a better way to do it.  In the mean time, 
that's really up to them to figure out and not something we can standarize 
around.

> > Low quality aggregation is not helpful. They key is for providers to do
> > it well, but that's not very related to anything that can be
> > standardized.
> 

We see in the IP reputation space plenty of examples of poorly run RBLs and a 
relative few that are really high quality and suitable for broad production 
use.  I expect that domain reputation will be similar, but trying to judge 
which reputation providers are good and which are bad is not something that 
can or should be centralized.  You may or may not be right, but I don't think 
it's anything to do with this discussion.

> message issued publicly on port 25.

I think that there is more than one way to do it and there is not any body of 
public experience on how best to do this.  It's pretty clear that the how of 
domain reputation calculation will remain largely private for some time, so 
there's no point in wasting time on standardizing pieces of it now.

Scott K