Journal of Computing Science and Engineering

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

A Survey of Transfer and Multitask Learning in Bioinformatics

  • Xu, Qian (Hong Kong University of Science and Technology) ;
  • Yang, Qiang (Hong Kong University of Science and Technology)
  • Received : 2011.07.18
  • Accepted : 2011.08.30
  • Published : 2011.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Machine learning and data mining have found many applications in biological domains, where we look to build predictive models based on labeled training data. However, in practice, high quality labeled data is scarce, and to label new data incurs high costs. Transfer and multitask learning offer an attractive alternative, by allowing useful knowledge to be extracted and transferred from data in auxiliary domains helps counter the lack of data problem in the target domain. In this article, we survey recent advances in transfer and multitask learning for bioinformatics applications. In particular, we survey several key bioinformatics application areas, including sequence classification, gene expression data analysis, biological network reconstruction and biomedical applications.

Keywords

References

  1. P. Larranaga, B. Calvo, R. Santana, C. Bielza, J. Galdiano, I. Inza, J. A. Lozano, R. Armananzas, G. Santafe, A. Perez, and V. Robles, "Machine learning in bioinformatics," Brief Bioinformatics, vol. 7, no. 1, pp. 86-112, Mar. 2006. https://doi.org/10.1093/bib/bbk007
  2. R. A. Caruana, "Multitask learning: a knowledge-based source of inductive bias," Proceedings of theTenth International Conference on Machine Learning, Amherst, MA, 1993, pp. 41-48.
  3. S. J. Pan and Q. Yang, "A survey on transfer learning," IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345-1359, 2010. https://doi.org/10.1109/TKDE.2009.191
  4. J. Blitzer, R. Mcdonald, and F. Pereira, "Domain adaptation with structural correspondence learning," Proceedings of the Conference on Empirical Methods in Natural Language Processing, Sydney, Australia, 2006, pp. 120-128.
  5. C. B. Do and A. Y. Ng, "Transfer learning for text classification," Proceedings of the 19th Annual Conference on Neural Information Processing Systems, Vancouver, Canada, 2005.
  6. W. Dai, Q. Yang, G. R. Xue, and Y. Yu, "Boosting for transfer learning," Proceedings of the 24th International Conference on Machine Learning, Corvalis, OR, 2007, pp. 193-200.
  7. P. C. Woodland, "Speaker adaptation for continuous density HMMS: a review," ISCA Tutorial and Research Workshop on Adaptation Methods for Speech Recognition, Sophia Antipolis, France, 2001.
  8. L. Xiao and J. Bilmes, "Regularized adaptation of discriminative classifiers," IEEE International Conference on Acoustics, Speech and Signal Processing, Toulouse, France, 2006, pp. I237-I240. https://doi.org/10.1109/ICASSP.2006.1660001
  9. R. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng, "Selftaught learning: transfer learning from unlabeled data," Proceedings of the 24th International Conference on Machine Learning, Corvalis, OR, 2007, pp. 759-766.
  10. J. Yang, R. Yan, and A. G. Hauptmann, "Cross-domain video concept detection using adaptive SVMs," Proceedings of the 15th ACM International Conference on Multimedia, Augsburg, Germany, 2007, pp. 188-197.
  11. V. W. Zheng, D. H. Hu, and Q. Yang, "Cross-domain activity recognition," Proceedings of the 11th ACM International Conference on Ubiquitous Computing, Orlando, FL, 2009, pp. 61-70.
  12. H. Y. Wang, V. W. Zheng, J. Zhao, and Q. Yang, "Indoor localization in multi-floor environments with reduced effort," Proceedings of the 8th IEEE International Conference on Pervasive Computing and Communications, Mannheim, Germany, 2010, pp. 244-252. https://doi.org/10.1109/PERCOM.2010.5466971
  13. P. Donnes and A. Elofsson, "Prediction of MHC class I binding peptides, using SVMHC," BMC Bioinformatics, vol. 3 pp. 25, Sep. 2002. https://doi.org/10.1186/1471-2105-3-25
  14. K. Aas, Microarray Data Mining: A Survey, Oslo, Norway: Norwegian Computing Center, 2001.
  15. E. P. Xing, M. I. Jordan, and R. M. Karp, "Feature selection for high-dimensional genomic microarray data," Proceedings of the 8th International Conference on Machine Learning, Williamstown, MA, 2001, pp. 601-608.
  16. W. H. Yang, D. Q. Dai, and H. Yan, "Finding correlated biclusters from gene expression data," IEEE Transactions on Knowledge and Data Engineering, vol. 23, no. 4, pp. 568-584, 2011. https://doi.org/10.1109/TKDE.2010.150
  17. C. Yang, Z. He, X. Wan, Q. Yang, H. Xue, and W. Yu, "SNPHarvester: a filtering-based approach for detecting epistatic interactions in genome-wide association studies," Bioinformatics, vol. 25, no. 4, pp. 504-511, Feb. 2009. https://doi.org/10.1093/bioinformatics/btn652
  18. X. Wan, C. Yang, Q. Yang, H. Xue, N. L. Tang, and W. Yu, "MegaSNPHunter: a learning approach to detect disease predisposition SNPs and high level interactions in genome wide association study," BMC Bioinformatics, vol. 10 pp. 13, 2009. https://doi.org/10.1186/1471-2105-10-13
  19. M. Krallinger and A. Valencia, "Text-mining and informationretrieval services for molecular biology," Genome Biology, vol. 6, no. 7, pp. 224, 2005. https://doi.org/10.1186/gb-2005-6-7-224
  20. C. Orwat, A. Graefe, and T. Faulwasser, "Towards pervasive computing in health care: a literature review," BMC Medical Informatics and Decision Making, vol. 8 pp. 26, 2008. https://doi.org/10.1186/1472-6947-8-26
  21. L. Bao and S. S. Intille, "Activity recognition from user-annotated acceleration data," Pervasive Computing: Second International Conference, PERVASIVE 2004, Linz/Vienna, Austria, April 21-23, 2004. Proceedings. Lectures Note in Computer Science Vol. 3001, A. Ferscha and F. Mattern, Eds., Heidelberg: Springer Berlin, 2004, pp. 1-17.
  22. N. Ravi, N. D, P. Mysore, and M. L. Littman, "Activity recognition from accelerometer data," Proceedings of the 17th Conference on Innovative Applications of Artificial Intelligence, Pittsburgh, PA, 2005, pp. 1541-1546.
  23. J. Yin, X. Chai, and Q. Yang, "High-level goal recognition in a wireless LAN," Proceedings of the 19th National Conference on Artificial Intelligence; Sixteenth Innovative Applications of Artificial Intelligence Conference, San Jose, CA, 2004, pp. 578-583.
  24. S. J. Cho, J. K. Oh, W. C. Bang, W. Chang, E. Choi, Y. Jing, J. Cho, and D. Y. Kim, "Magic wand: a hand-drawn gesture input device in 3-D space with inertial sensors," Ninth International Workshop on Frontiers in Handwriting Recognition, Tokyo, Japan, 2004, pp. 106-111. https://doi.org/10.1109/IWFHR.2004.66
  25. M. Chen, B. Huang, and Y. Xu, "Human abnormal gait modeling via hidden Markov model," Proceedings of the 2007 International Conference on Information Acquisition, Jeju, Korea, 2007, pp. 517-522.
  26. S. D. Choi, A. S. Lee, and S. Y. Lee, "On-line handwritten character recognition with 3D accelerometer," Proceedings of the IEEE International Conference on Information Acquisition, Weihai, China, 2006, pp. 845-850. https://doi.org/10.1109/ICIA.2006.305842
  27. L. Nachman, A. Baxi, S. Bhattacharya, V. Darera, P. Deshpande, N. Kodalapura, V. Mageshkumar, S. Rath, J. Shahabdeen, and R. Acharya, "Jog falls: a pervasive healthcare platform for diabetes management," Pervasive Computing. Lecture Notes in Computer Science Vol. 6030, P. Floreen, A. Kruger, and M. Spasojevic, Eds., Heidelberg: Springer Berlin, pp. 94-111, 2010. https://doi.org/10.1007/978-3-642-12654-3_6
  28. S. Consolvo, K. Everitt, I. Smith, and J. A. Landay, "Design requirements for technologies that encourage physical activity," Conference on Human Factors in Computing Systems, Montreal, Canada, 2006, pp. 457-466.
  29. T. Evgeniou and M. Pontil, "Regularized multi-task learning," Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, 2004, pp. 109-117.
  30. C. Widmer, J. Leiva, Y. Altun, and G. Ratsch, "Leveraging sequence classification by taxonomy-based multitask learning," Research in Computational Molecular Biology: 14th Annual International Conference, RECOMB 2010, Lisbon, Portugal, April 25-28, 2010. Proceedings. Lecture Notes in Computer Science Vol. 6044, B. Berger, Ed., Heidelberg: Springer Berlin, pp. 522-534, 2010.
  31. G. Schweikert, C. Widmer, B. Scholkopf, and G. Ratsch, "An empirical analysis of domain adaptation algorithms for genomic sequence," Proceedings of the 22nd Annual Conference on Neural Information Processing Systems, Vancouver, Canada, 2008.
  32. L. Jacob and J. P. Vert, "Efficient peptide-MHC-I binding prediction for alleles with few known binders," Bioinformatics, vol. 24, no. 3, pp. 358-366, Feb. 2007.
  33. L. Jacob, F. Bach, and J. P. Vert, "Clustered multi-task learning: a convex formulation," Proceedings of the 23th Annual Conference on Neural Information Processing Systems, Vancouver, Canada, 2009.
  34. C. Widmer, N. Toussaint, Y. Altun, O. Kohlbacher, and G. Ratsch, "Novel machine learning methods for MHC class I binding prediction," Pattern Recognition in Bioinformatics: 5th IAPR International Conference, PRIB 2010, Nijmegen, The Netherlands, September 22-24, 2010. Proceedings. Lecture Notes in Computer Science Vol. 6282, T. Dijkstra, E. Tsivtsivadze, E. Marchiori, and T. Heskes, Eds., Heidelberg: Springer Berlin, pp. 98-109, 2010.
  35. C. Widmer, N. C. Toussaint, Y. Altun, and G. Ratsch, "Inferring latent task structure for Multitask Learning by Multiple Kernel Learning," BMC Bioinformatics, vol. 11 Suppl 8 pp. S5, 2010.
  36. Q. Xu, S. J. Pan, H. H. Xue, and Q. Yang, "Multitask learning for protein subcellular location prediction," IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 8, no. 3, pp. 748-759, 2011. https://doi.org/10.1109/TCBB.2010.22
  37. A. Argyriou, T. Evgeniou, and M. Pontil, "Multi-task feature learning," Advances in Neural Information Processing Systems Vol. 19: Proceedings of the 2006 Neural Information Processing System Conference, B. Scholkopf, J. Platt, and T. Hofmann, Eds., Cambridge, MA: MIT Press, 2007, pp. 41-48.
  38. A. Argyriou, T. Evgeniou, and M. Pontil, "Convex multi-task feature learning," Machine Learning, vol. 73, no. 3, pp. 243-272, 2008. https://doi.org/10.1007/s10994-007-5040-8
  39. Q. Liu, Q. Xu, V. W. Zheng, H. Xue, Z. Cao, and Q. Yang, "Multi-task learning for cross-platform siRNA efficacy prediction: an in-silico study," BMC Bioinformatics, vol. 11 pp. 181, 2010. https://doi.org/10.1186/1471-2105-11-181
  40. A. Chen and Z. W. Huang, "A new multi-task learning technique to predict classification of leukemia and prostate cancer," Medical Biometrics. Lecture Notes in Computer Science Vol. 6165, D. Zhang and M. Sonka, Eds., Heidelberg: Springer Berlin, pp. 11-20, 2010. https://doi.org/10.1007/978-3-642-13923-9_2
  41. K. Puniyani, S. Kim, and E. P. Xing, "Multi-population GWA mapping via multi-task regularized regression," Bioinformatics, vol. 26, no. 12, pp. i208-i216, Jun. 2010. https://doi.org/10.1093/bioinformatics/btq191
  42. Q. Xu, H. Xue, and Q. Yang, "Multi-platform gene expression mining and marker gene analysis," International Journal of Data Mining and Bioinformatics 2010 in press.
  43. Y. Tamada, H. Bannai, S. Imoto, T. Katayama, M. Kanehisa, and S. Miyano, "Utilizing evolutionary information and gene expression data for estimating gene networks with Bayesian network models," Journal of Bioinformatics and Computational Biology, vol. 3, no. 6, pp. 1295-1313, 2005. https://doi.org/10.1142/S0219720005001569
  44. M. Nassar, R. Abdallah, H. A. Zeineddine, E. Yaacoub, and Z. Dawy, "A new multitask learning method for multiorganism gene network estimation," IEEE International Symposium on Information Theory, Toronto, Canada, 2008, pp. 2287-2291. https://doi.org/10.1109/ISIT.2008.4595398
  45. T. Kato, K. Okada, H. Kashima, and M. Sugiyama, "A transfer learning approach and selective integration of multiple types of assays for biological network inference," International Journal of Knowledge Discovery in Bioinformatics, vol. 1, no. 1, pp. 66-80, Jan.-Mar. 2010. https://doi.org/10.4018/jkdb.2010100205
  46. Y. Qi, O. Tastan, J. G. Carbonell, J. Klein-Seetharaman, and J. Weston, "Semi-supervised multi-task learning for predicting interactions between HIV-1 and human proteins," Bioinformatics, vol. 26, no. 18, pp. i645-i652, 2010. https://doi.org/10.1093/bioinformatics/btq394
  47. Q. Xu, E. W. Xiang, and Q. Yang, "Protein-protein interaction prediction via collective matrix factorization," Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine, Hong Kong, 2010, pp. 62-67. https://doi.org/10.1109/BIBM.2010.5706537
  48. A. P. Singh and G. J. Gordon, "Relational learning via collective matrix factorization," Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, NV, 2008, pp. 650-658.
  49. K. Zhang, J. W. Gray, and B. Parvin, "Sparse multitask regression for identifying common mechanism of response to therapeutic targets," Bioinformatics, vol. 26, no. 12, pp. i97-i105, Jun. 2010. https://doi.org/10.1093/bioinformatics/btq181
  50. S. Bickel, J. Bogojeska, T. Lengauer, and T. Scheffer, "Multi-task learning for HIV therapy screening," Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, 2008, pp. 56-63.
  51. D. Dahlmeier and H. T. Ng, "Domain adaptation for semantic role labeling in the biomedical domain," Bioinformatics, vol. 26, no. 8, pp. 1098-1104, 2010. https://doi.org/10.1093/bioinformatics/btq075
  52. J. Jiang and C. Zhai, "Instance weighting for domain adaptation in NLP," Proceedings of the 45th Annual Meeting of the Associationfor Computational Linguistics, Prague, Czech Republic, 2007, pp. 264-271.
  53. H. Daume III, "Frustratingly easy domain adaptation," Proceedings of the 45th Annual Meeting of the Associationfor Computational Linguistics, Prague, Czech Republic, 2007, pp. 256-263.
  54. J. Bi, T. Xiong, S. Yu, M. Dundar, and R. Rao, "An improved multi-task learning approach with applications in medical diagnosis," Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008, Antwerp, Belgium, September 15-19, 2008, Proceedings, Part I. Lecture Notes in Computer Science Vol. 5211, W. Daelemans, B. Goethals, and K. Morik, Eds., Heidelberg: Springer Berlin, pp. 117-132, 2008.
  55. P. Rashidi and D. J. Cook, "Transferring learned activities in smart environments," Proceedings of the 5th International Conference on Intelligent Environments, Barcelona, Spain, 2009, pp. 185-192.
  56. T. L. M. Van Kasteren, G. Englebienne, and B. J. A. Krose, "Recognizing activities in multiple contexts using transfer learning," AAAI Fall Symposium Technical Report Vol. FS-08-02, Arlington, VA, 2008, pp. 142-149.
  57. P. Rashidi and D. J. Cook, "Home to home transfer learning," Proceedings of the AAAI Plan, Activity, and Intent Recognition Workshop, Atlanta, GA, 2010.
  58. P. Rashidi and D. J. Cook, "Multi home transfer learning for resident activity discovery and recognition," Proceedings of the 4th International Workshop on Knowledge Discovery from Sensor Data, Washington, DC, 2010.
  59. P. T. Spellman, G. Sherlock, M. Q. Zhang, V. R. Iyer, K. Anders, M. B. Eisen, P. O. Brown, D. Botstein, and B. Futcher, "Comprehensive identification of cell cycle-regulated genes of the yeast Saccharomyces cerevisiae by microarray hybridization," Molecular Biology of the Cell, vol. 9, no. 12, pp. 3273-3297, Dec. 1998. https://doi.org/10.1091/mbc.9.12.3273
  60. M. L. Whitfield, G. Sherlock, A. J. Saldanha, J. I. Murray, C. A. Ball, K. E. Alexander, J. C. Matese, C. M. Perou, M. M. Hurt, P. O. Brown, and D. Botstein, "Identification of genes periodically expressed in the human cell cycle and their expression in tumors," Molecular Biology of the Cell, vol. 13, no. 6, pp. 1977-2000, Jun. 2002. https://doi.org/10.1091/mbc.02-02-0030.
  61. Y. Yamanishi, M. Araki, A. Gutteridge, W. Honda and M. Kanehisa, "Prediction of drug-target interaction networks from the integration of chemical and genomic spaces", Bioinformatics, vol. 24, no. 13, pp. i232-i240, 2008. https://doi.org/10.1093/bioinformatics/btn162
  62. H. Kashima, Y. Yamanishi, T. Kato, M. Sugiyama, and K. Tsuda, "Simultaneous inference of biological networks of multiple species from genome-wide data and evolutionary information: a semi-supervised approach," Bioinformatics, vol. 25, no. 22, pp. 2962-2968, Nov. 2009. https://doi.org/10.1093/bioinformatics/btp494
  63. S. Mei, W. Fei, and S. Zhou, "Gene ontology based transfer learning for protein subcellular localization," BMC Bioinformatics, vol. 12 pp. 44, 2011. https://doi.org/10.1186/1471-2105-12-44
  64. L. Jacob and J. P. Vert, "Protein-ligand interaction prediction: an improved chemogenomics approach," Bioinformatics, vol. 24, no. 19, pp. 2149-2156, Oct. 2008. https://doi.org/10.1093/bioinformatics/btn409
  65. Z. Zhao, Y. Chen, J. Liu, and M. Liu, "Cross-mobile ELM based activity recognition," International Journal of Engineering and Industries, vol. 1, no. 1, pp. 30-38, Dec. 2010. https://doi.org/10.4156/ijei.vol1.issue1.3

Cited by

  1. Fuzzy Transfer Learning: Methodology and application vol.293, 2015, https://doi.org/10.1016/j.ins.2014.09.004
  2. A hybrid approach to automatic de-identification of psychiatric notes 2017, https://doi.org/10.1016/j.jbi.2017.06.006
  3. Domain adaptation for semantic role labeling of clinical text vol.22, pp.5, 2015, https://doi.org/10.1093/jamia/ocu048
  4. Knowledge-transfer learning for prediction of matrix metalloprotease substrate-cleavage sites vol.7, pp.1, 2017, https://doi.org/10.1038/s41598-017-06219-7
  5. Multitask learning for host–pathogen protein interactions vol.29, pp.13, 2013, https://doi.org/10.1093/bioinformatics/btt245
  6. Statistical transfer learning: A review and some extensions to statistical process control 2018, https://doi.org/10.1080/08982112.2017.1373810
  7. Review of computational methods for virus–host protein interaction prediction: a case study on novel Ebola–human interactions 2017, https://doi.org/10.1093/bfgp/elx026
  8. Structured Data Fusion vol.9, pp.4, 2015, https://doi.org/10.1109/JSTSP.2015.2400415
  9. Computational approaches for prediction of pathogen-host protein-protein interactions vol.6, 2015, https://doi.org/10.3389/fmicb.2015.00094
  10. Fuzzy Regression Transfer Learning in Takagi–Sugeno Fuzzy Models vol.25, pp.6, 2017, https://doi.org/10.1109/TFUZZ.2016.2633376
  11. Probabilistic Low-Rank Multitask Learning vol.29, pp.3, 2018, https://doi.org/10.1109/TNNLS.2016.2641160
  12. : Persistent homology-based multi-task deep neural networks for simultaneous predictions of partition coefficient and aqueous solubility vol.39, pp.20, 2018, https://doi.org/10.1002/jcc.25213