Training and testing binding thrombin dataset

+The test set consists of 634 data points, each of which represents
+a molecule that is either active (A) or inactive (I).  The test set
+has the same format as the training set, with the exception that the
+activity value (A or I) for each data point is missing, that is, has
+been replaced by a question mark (?).  Please submit one prediction,
+A or I, for each data point.  Your submission should be in the form
+of a file that starts with your contact information, followed by a
+line with 5 asterisks, followed immediately by your predictions, with
+one line per data point.  The predictions should be in the same order
+as the test set data points.  So your prediction for the first example
+should appear on the first line after the asterisks, your prediction
+for the second example should appear on the second line after the
+asterisks, etc.  Hence, after your contact information, the prediction
+file will consist of 635 lines and have the form:
+
+*****
+I
+I
+A
+I
+A
+I
+
+etc.
+
+You may submit your prediction by email to page@biostat.wisc.edu
+or by anonymous ftp to ftp.biostat.wisc.edu, placing the file
+into the directory dropboxes/page/.  If using email, please use
+the subject line "KDDcup <name> thrombin" where <name> is your
+name.  If using ftp, please name the file KDDcup.<name>.thrombin
+where <name> is your name.  For example, my submission would be
+named KDDcup.DavidPage.thrombin
+
+Only one submission per person per task is permitted.  If you do not
+receive email confirmation of your submission within 24 hours, please
+email page@biostat.wisc.edu with subject "KDDcup no confirmation".
+
+For group entries, the contact information should include the names
+of everyone to be credited as a member of the group should your entry
+achieve the highest score.  But no person is to be listed on more than
+one entry per task.

+Prediction of Molecular Bioactivity for Drug Design -- Binding to Thrombin
+--------------------------------------------------------------------------
+
+Drugs are typically small organic molecules that achieve their desired
+activity by binding to a target site on a receptor. The first step in
+the discovery of a new drug is usually to identify and isolate the
+receptor to which it should bind, followed by testing many small
+molecules for their ability to bind to the target site. This leaves
+researchers with the task of determining what separates the active
+(binding) compounds from the inactive (non-binding) ones.  Such a
+determination can then be used in the design of new compounds that not
+only bind, but also have all the other properties required for a drug
+(solubility, oral absorption, lack of side effects, appropriate duration
+of action, toxicity, etc.). 
+
+The present training data set consists of 1909 compounds tested for
+their ability to bind to a target site on thrombin, a key receptor in
+blood clotting. The chemical structures of these compounds are not
+necessary for our analysis and are not included. Of these compounds, 42
+are active (bind well) and the others are inactive. Each compound is
+described by a single feature vector comprised of a class value (A for
+active, I for inactive) and 139,351 binary features, which describe
+three-dimensional properties of the molecule. The definitions of the
+individual bits are not included - we don't know what each individual
+bit means, only that they are generated in an internally consistent
+manner for all 1909 compounds. Biological activity in general, and
+receptor binding affinity in particular, correlate with various
+structural and physical properties of small organic molecules. The task
+is to determine which of these properties are critical in this case and
+to learn to accurately predict the class value.  To simulate the
+real-world drug design environment, the test set contains 636 additional
+compounds that were in fact generated based on the assay results
+recorded for the training set. In evaluating the accuracy, a
+differential cost model will be used, so that the sum of the costs of
+the actives will be equal to the sum of the costs of the inactives.
+
+We thank DuPont Pharmaceuticals for graciously providing this data set
+for the KDD Cup 2001 competition.  All publications referring to
+analysis of this data set should acknowledge DuPont Pharmaceuticals
+Research Laboratories and KDD Cup 2001.

+I
+A
+I
+I
+I
+A
+I
+I
+I
+A
+I
+I
+I
+A
+I
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+A
+I
+I
+I
+I
+I
+A
+I
+I
+I
+A
+I
+I
+I
+I
+I
+I
+I
+I
+A
+A
+I
+I
+I
+I
+I
+A
+I
+A
+A
+I
+I
+I
+A
+I
+I
+I
+I
+A
+A
+I
+A
+I
+I
+A
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+I
+A
+I
+A
+I
+I
+I
+A
+A
+I
+I
+I
+I
+I
+I
+A
+I
+I
+I
+I
+A
+A
+I
+I
+I
+I
+I
+I
+I
+I
+A
+A
+I
+A
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+I
+I
+I
+A
+I
+I
+I
+I
+I
+I
+I
+A
+I
+I
+A
+I
+A
+I
+I
+A
+I
+A
+I
+A
+I
+A
+I
+I
+I
+I
+I
+A
+I
+I
+A
+I
+I
+A
+I
+I
+I
+A
+I
+A
+I
+I
+A
+I
+I
+I
+I
+A
+I
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+A
+I
+I
+I
+I
+I
+I
+A
+I
+I
+A
+A
+A
+I
+I
+A
+A
+I
+I
+I
+I
+A
+I
+I
+I
+I
+A
+I
+A
+I
+I
+I
+I
+I
+I
+I
+A
+A
+I
+I
+I
+I
+I
+I
+I
+I
+A
+A
+I
+I
+I
+I
+I
+I
+A
+A
+I
+I
+I
+I
+I
+I
+A
+I
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+I
+A
+I
+I
+I
+I
+I
+I
+A
+A
+I
+I
+I
+I
+I
+A
+I
+I
+I
+I
+I
+A
+A
+A
+I
+A
+I
+I
+I
+I
+A
+A
+I
+A
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+I
+I
+I
+A
+A
+I
+I
+A
+I
+I
+I
+I
+I
+A
+A
+I
+A
+I
+I
+I
+I
+I
+I
+A
+A
+I
+I
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+I
+A
+I
+I
+A
+I
+I
+I
+I
+A
+A
+I
+A
+A
+I
+I
+A
+I
+I
+I
+I
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+I
+A
+A
+I
+I
+I
+A
+I
+I
+I
+I
+A
+I
+A
+I
+I
+I
+I
+I
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+A
+A
+A
+I
+I
+I
+A
+A
+I
+I
+I
+I
+I
+A
+I
+A
+I
+I
+I
+I
+I
+A
+I
+I
+I
+I
+A
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+A
+I
+I
+A
+I
+I
+I
+I
+A
+I
+I
+A
+A
+I
+I
+I
+A
+I
+A
+I
+I
+I
+I
+I
+I
+I
+A
+A
+I
+I
+I
+A
+I
+I
+I
+A
+I
+I
+I
+I
+I
+I
+A
+I
+I
+I
+I
+I
+A
+I
+I
+I
+I
+I
+A
+I
+I
+A
+I
+I
+I
+I
+I
+I
+A
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+I
+A
+I
+I
+A
+I
+I
+I
+I
+A
+I
+I
+I
+I
+I
+A
+I
+I
+I
+A
+I
+I
+I
+A
+I
+A
+A
+I
+A
+I
+I
+I
+I
+I
+I
+I
+A
+I
+I
+I
+A
+I
+A
+I
+I
+I
+I
+A
+I
+A
+I

+# -*- encoding: utf-8 -*-
+
+import os
+from time import time
+import argparse
+from sklearn.naive_bayes import BernoulliNB
+from sklearn.svm import SVC
+from sklearn.neighbors import NearestCentroid
+from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrix, \
+    classification_report
+import sys
+from scipy.sparse import csr_matrix
+import numpy as np
+
+__author__ = 'CMendezC'
+
+# Goal: training and validation binding thrombin data set
+
+# Parameters:
+# 1) --inputPath Path to read input files.
+# 2) --inputTrainingData File to read training data.
+# 3) --inputTestingData File to read testing data.
+# 4) --inputTestingClasses File to read testing classes.
+# 5) --outputModelPath Path to place output model.
+# 6) --outputModelFile File to place output model.
+# 7) --outputReportPath Path to place evaluation report.
+# 8) --outputReportFile File to place evaluation report.
+# 9) --classifier Classifier: BernoulliNB, SVM, NearestCentroid.
+
+# Ouput:
+# 1) Classification model and evaluation report.
+
+# Execution:
+
+# python training-validation-binding-thrombin.py
+# --inputPath /home/compu2/bionlp/lcg-bioinfoI-bionlp/binding-thrombin-dataset
+# --inputTrainingData thrombin.data
+# --inputTestingData Thrombin.testset
+# --inputTestingClasses Thrombin.testset.class
+# --outputModelPath /home/compu2/bionlp/lcg-bioinfoI-bionlp/binding-thrombin-dataset/models
+# --outputModelFile SVM-model.mod
+# --outputReportPath /home/compu2/bionlp/lcg-bioinfoI-bionlp/binding-thrombin-dataset/reports
+# --outputReportFile SVM.txt
+# --classifier SVM
+
+# source activate python3
+
+###########################################################
+#                       MAIN PROGRAM                      #
+###########################################################
+
+if __name__ == "__main__":
+    # Parameter definition
+    parser = argparse.ArgumentParser(description='Training validation Binding Thrombin Dataset.')
+    parser.add_argument("--inputPath", dest="inputPath",
+                      help="Path to read input files", metavar="PATH")
+    parser.add_argument("--inputTrainingData", dest="inputTrainingData",
+                      help="File to read training data", metavar="FILE")
+    parser.add_argument("--inputTestingData", dest="inputTestingData",
+                      help="File to read testing data", metavar="FILE")
+    parser.add_argument("--inputTestingClasses", dest="inputTestingClasses",
+                      help="File to read testing classes", metavar="FILE")
+    parser.add_argument("--outputModelPath", dest="outputModelPath",
+                      help="Path to place output model", metavar="PATH")
+    parser.add_argument("--outputModelFile", dest="outputModelFile",
+                      help="File to place output model", metavar="FILE")
+    parser.add_argument("--outputReportPath", dest="outputReportPath",
+                      help="Path to place evaluation report", metavar="PATH")
+    parser.add_argument("--outputReportFile", dest="outputReportFile",
+                      help="File to place evaluation report", metavar="FILE")
+    parser.add_argument("--classifier", dest="classifier",
+                      help="Classifier", metavar="NAME",
+                      choices=('BernoulliNB', 'SVM', 'NearestCentroid'), default='SVM')
+
+    (options, args) = parser.parse_args()
+    if len(args) > 0:
+        parser.error("None parameters indicated.")
+        sys.exit(1)
+
+    # Printing parameter values
+    print('-------------------------------- PARAMETERS --------------------------------')
+    print("Path to read input files: " + str(options.inputPath))
+    print("File to read training data: " + str(options.inputTrainingData))
+    print("File to read testing data: " + str(options.inputTestingData))
+    print("File to read testing classes: " + str(options.inputTestingClasses))
+    print("Path to place output model: " + str(options.outputModelPath))
+    print("File to place output model: " + str(options.outputModelFile))
+    print("Path to place evaluation report: " + str(options.outputReportPath))
+    print("File to place evaluation report: " + str(options.outputReportFile))
+    print("Classifier: " + str(options.outputFile))
+
+    # Start time
+    t0 = time()
+
+    print("   Reading training data and true classes...")
+    trainingClasses = []
+    trainingData = []
+    with open(os.path.join(options.inputPath, options.inputTrainingData), encoding='utf8', mode='r') \
+            as iFile:
+        for line in iFile:
+            line = line.strip('\r\n')
+            listLine = line.split(',')
+            trainingClasses.append(listLine[0])
+            trainingData.append(listLine[1:])
+    # trainingMatrix = np.matrix(trainingData)
+    trainingMatrix = csr_matrix(trainingData, dtype='double')
+
+    print("Number of training classes: {}".format(len(trainingClasses)))
+    print("Number of training class A: {}".format(trainingClasses.count('A')))
+    print("Number of training class I: {}".format(trainingClasses.count('I')))
+    print("Shape of training matrix: {}".format(trainingMatrix.shape))
+
+    print("   Reading testing data and true classes...")
+    testingClasses = []
+    testingData = []
+    with open(os.path.join(options.inputPath, options.inputTestingData), encoding='utf8', mode='r') \
+            as iFile:
+        for line in iFile:
+            line = line.strip('\r\n')
+            listLine = line.split(',')
+            testingData.append(listLine)
+    testingMatrix = csr_matrix(testingData, dtype='double')
+    with open(os.path.join(options.inputPath, options.inputTestingClasses), encoding='utf8', mode='r') \
+            as iFile:
+        for line in iFile:
+            line = line.strip('\r\n')
+            testingClasses.append(line)
+
+    print("Number of testing classes: {}".format(len(testingClasses)))
+    print("Number of testing class A: {}".format(trainingClasses.count('A')))
+    print("Number of testing class I: {}".format(trainingClasses.count('I')))
+    print("Shape of testing matrix: {}".format(testingMatrix.shape))
+
+    if options.classifier == "MultinomialNB":
+        classifier = BernoulliNB()
+    elif options.classifier == "SVM":
+        classifier = SVC()
+    elif options.classifier == "NearestCentroid":
+        classifier = NearestCentroid()
+
+    print("   Training...")
+    classifier.fit(trainingMatrix, trainingClasses)
+    print("      Done!")
+
+    print("   Testing (prediction in new data)...")
+    y_pred = classifier.predict(testingMatrix)
+    print("      Done!")
+
+    print("   Saving report...")
+    with open(os.path.join(options.outputPath, options.outputFile), mode='w', encoding='utf8') as oFile:
+        oFile.write('**********        EVALUATION REPORT     **********\n')
+        oFile.write('Classifier: {}\n'.format(options.classifier))
+        oFile.write('Accuracy: {}\n'.format(accuracy_score(testingClasses, y_pred)))
+        oFile.write('Precision: {}\n'.format(precision_score(testingClasses, y_pred, average='weighted')))
+        oFile.write('Recall: {}\n'.format(recall_score(testingClasses, y_pred, average='weighted')))
+        oFile.write('F-score: {}\n'.format(f1_score(testingClasses, y_pred, average='weighted')))
+        oFile.write('Confusion matrix: \n')
+        oFile.write(str(confusion_matrix(testingClasses, y_pred)) + '\n')
+        oFile.write('Classification report: \n')
+        oFile.write(classification_report(testingClasses, y_pred) + '\n')
+    print("      Done!")
+
+    print("Training and testing done in: %fs" % (time() - t0))