Rainbow flags could be taken from fans at the World Cup in Qatar to protect them from being attacked for promoting gay rights, a senior leader overseeing security for the tournament told The Associated Press.

Major General Abdulaziz Abdullah Al Ansari insisted that LGBTQ couples would be welcomed and accepted in Qatar for the Nov. 21-Dec. 18 FIFA showpiece despite same-sex relations remaining criminalized in the conservative Gulf nation.

But Al Ansari is against the overt promotion of LGBTQ freedoms as symbolized by the rainbow flag that FIFA and World Cup organizers had previously said would be welcome across Qatar’s eight stadiums.

“If he (a fan) raised the rainbow flag and I took it from him, it’s not because I really want to, really, take it, to really insult him, but to protect him,” Al Ansari told the AP. “Because if it’s not me, somebody else around him might attack (him) … I cannot guarantee the behavior of the whole people. And I will tell him: ‘Please, no need to really raise that flag at this point.’”

Al Ansari is director of the Department of International Cooperation and Chairman of the National Counterterrorism Committee at the Ministry of Interior where he discussed World Cup planning for an hour with the AP.