Subject: [Asrg] 0. General- useful ideas from other areas of research.

From: "John Fenley" <[email-masked]>
Date: Mon, 29 Dec 2003 19:14:17 -0700

In science news vol. 164 #25&26 the main article entitled “Bookish Math” is 
about modern techniques for analyzing text, collectively called stylometry. 
Using stylometry on various anonymously written or disputed texts has given 
new data that has helped to resolve the true author. This may have impact on 
future content based filters.

The full text of the article “Bookish Math” may be found at the science news 
website
http://www.sciencenews.org

of particular interest to me are support-vector machines.
from the article:

“As does PCA[another technique mentioned], the new technique plots each 
chunk of text as a point in a high-dimensional space. It then searches for 
the best-fitting surface that divides the points belonging to one author 
from those of the other author.”

I mentioned this idea before, but did not know this actually existed for 
text classification.
http://article.gmane.org/gmane.ietf.asrg/6093

John Fenley

_________________________________________________________________
Make your home warm and cozy this winter with tips from MSN House & Home.  
http://special.msn.com/home/warmhome.armx

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Art Pollard <[email-masked]>
Date: Mon, 29 Dec 2003 21:26:31 -0700

>The full text of the article "Bookish Math" may be found at the science 
>news website
>from the article ....

The problem with Support Vector Machines (SVM's) is that they can take an 
incredibly long period of time to train.  Bayesian takes comparatively no 
time to train.  SVM's can take from minutes to hours to days to train 
depending on the number of features you train it on. (A feature is a item 
that you classify on such as the phrase "Make Money Fast" or "this is not 
spam.")  The more features and examples, the longer it takes to train SVMs 
as you might expect.  The problem is that when the time increases, it 
doesn't increase in a nice linear fashion as one might hope.

SVMs are great at classification.  The only thing is that you need to have 
the horsepower and/or time to train them.

-Art

-- 
Art Pollard
http://www.lextek.com/
Suppliers of High Performance Text Retrieval Engines.