The paper presents ongoing issues, challenges, and difficulties we face in applying machine learning methods to retrospectively collected clinical data. The objective of our research is to build a reliable prediction model for early assessment of emergency pediatric asthma exacerbations. This predictive model should be able to distinguish between patients with mild or moderate/severe asthma attacks at a medically acceptable level of performance. Our real-life data set presents us with some difficult challenges which we communicate in this paper. Our approach to overcoming some of these difficulties is to use external expert knowledge to aid with classification by decomposing the classification problem into a two-tier concept, where concepts can be explicitly described in terms of the external knowledge source. Such an approach also has the advantage of significantly reducing the size of the training set required.