I cannot answer for
Martin Escardo (npub14a5…9fse) but here's a definite issue:
Martin or myself have little to be scared of if visiting the US (except insurance issues should we have a medical problem, but let us not digress).
However, others (trans persons…) have more to fear. By organizing scientific events in the US, we may exclude these people from the event.
Thus there is a valid case for avoiding attending events there, so as to encourage organizers to schedule them elsewhere.