End-to-end Object Detection with Transformer (DETR) performs object detection with Transformer and achieves comparable performance with two-stage object detection like Faster-RCNN. However, DETR needs huge computational resources for training and inference due to the high-resolution spatial inputs. In this paper, a novel variant of transformer named Adaptive Clustering Transformer (ACT) has been proposed to reduce the computation cost for high-resolution input. ACT clusters the query features adaptively using Locality Sensitive Hashing (LSH) and approximates the query-key interaction using the prototype-key interaction. ACT can reduce the quadratic O(