More than once I have explained my reservations about accepting a calling (and I mean given detailed explanations about why I was hesitant) - with the caveat that I would accept it if they went back and talked and prayed about it and still felt they needed me to fill the position.
Once I explained that if I did accept it I would be "filling a slot" and probably wouldn't be able to "magnify" anything, given my work situation at the time.
One of those times, they still asked me to accept the calling, so I did - and was released eventually when someone better was able to do it.
Twice, they talked about it and never extended the calling.
Finally, once I was in a holding pattern for an extended period of time with a potential calling based on what I shared with the person who spoke with me. It actually is a calling I would have liked to accept, but I wasn't sure if I could do it well or for how long - since we thought we might have been moving in the near future. They waited for us to decide fully about that possible move, then the calling was extended when we decided not to move.
I share all of that to say this:
To those being extended a calling:
If you aren't sure about accepting a particular calling, be open and honest about your concerns with those who are talking with you about that calling. Be willing to accept a calling with which you aren't comfortable, but don't withhold important information if those who are talking with you aren't aware of it. Don't refuse reflexively - but if you can't accept reflexively, that's fine, as well. Talk about it; don't hide your feelings and concerns.
To those extending a calling:
Talk with the person first, before actually extending the calling. Be willing to reconsider, talk again and pray again about it if the person has concerns. You are dealing with real people, so give them sincere consideration as agents unto themselves. Respect their agency - and truly listen to them.