Tax administrations globally increasingly rely on artificial intelligence (AI) systems for automation. However, automation has a huge potential impact on the rights of taxpayers subject to algorithmic assessments, which is compounded by the opacity of complex AI systems. This article argues that adequate protection of taxpayers’ rights demands the use of eXplainable AI (XAI) technologies that can render the functioning and decisions of tax AI systems understandable for taxpayers, administrative appeal bodies and the courts. This demand follows from the constitutional principles that guide taxation. Still, it is insufficiently addressed by soft and hard law instruments on AI, which do not address the particular information needs of the tax domain. To address this gap, the authors conclude the article by mapping technical and legal challenges for the proper application of explanation techniques to tax AI to ensure that automation does not come at the expense of taxpayers’ rights.