Simple pre-processing + text categorization + outlier analysis
a single dataset; choose before November 2

30 p. + extrapoints for interesting extension. Deadline: January 12

• Before: choose an optimal number (OPT) of attributes (check a learning curve)
• Use StringToWrdVector from Weka and
  • Stemmer: IteratedLovinsStemmer
  • Stop word handler: Rainbow
  • Build three document-term matrix, 0/1, Term Frequency and TF-IDF.
  • For 50%, 75& and 100% OPT do
• Use 10-cross validation and
• use all combinations of Outlier detection methods (OD, use OD from OD-default.json) and clasifiers (CLF, use the classifiers in CLF-default.json )
• OD: remove always 1%, 2% and 5% most outlying instances
• CLF: default settings

a report in English including

• a dataset and task description, data preparation (e.g. transformation into form)
  
• description of the method used including
  analysis of a learning curve (x=num. of attributes, y=accuracy)
  short description of pre-processing (PP), outlier detection (OD) and learning algorithms (LA)
  
• results
  
• comparison of different combinations of {PP,OD}+LA including
  graphs and statistical tests
  
• discussion including discussion of the outliers found
  conclusion
  
• link to all data and results

Task 2 : 20 p. Deadline: February 4

Advanced pre-processing (via a NLP tool) + text categorisation (exceptionally some other mining task)
in groups of 3..4, with the same datasets as in Task 1
a short single report per group
if excellent, with a report 10-12 pages, may replace a final exam.