We propose a new tagger for the Czech language and particu-larly for the tagset used for annotation of corpora of the TenTen family.The tagger is based on neural networks with pretrained word embed-dings. We selected the newest Czech Web corpus of the TenTen familyas training data, but we removed sentences with phenomena that wereoften annotated incorrectly. We let the tagger to learn the annotation ofthese phenomena on its own. We also experimented with the recognitionof multi-word expressions since this information can support the correcttagging.We evaluated the tagger on 6,950 sentences (84,023 tokens) from thecstenten17corpus and achieved 75.25% accuracy when compared bytags. When compared by attributes, we achieved 91.62% accuracy; theaccuracy of POS tag prediction is 96.5%.