Yes, your phone should be off, but it can still be tracked while offline. Hence, the importance of the blocker bag.
ALT
You should account for the fact that your phone may get lost, stolen, or broken. There’s also a risk of your phone being confiscated by authorities — which means that if they’re able to unlock your phone, they’ll have access to data on you and people you know. It could give authorities access to information about what is being organized and who is doing the organizing and might even give them the information necessary to shut down or prevent protests and arrest those involved.
How to secure your phone before attending a protest / Protect your digital security
ALT