With global life expectancy rising every year, ageing-associated diseases are becoming an increasingly important problem. Very often, successful treatment relies on early diagnosis. In this work, the issue of Parkinson's disease (PD) diagnostics is tackled. It is particularly important, as there are no certain antemortem methods of diagnosing PD - meaning that the presence of the disease can only be confirmed after the patient's death. In our work, we propose a non-invasive approach for classification of raw speech recordings for PD recognition using deep learning models. The core of the method is an audio classifier using knowledge transfer from a pretrained natural language model, namely wav2vec 2.0. The model was tested on a group of 38 PD patients and 10 healthy persons above the age of 50. A dataset of speech recordings acquired using a smartphone recorder was constructed and the recordings were labelled as PD/non-PD with the severity of the disease additionally rated using Hoehn-Yahr scale. We then benchmarked the classification performance against baseline methods. Additionally, we show an assessment of human-level performance with neurology professionals.