You need to deploy a simple contract per user (who voted) and calculate address (deterministic) for store user vote. You main contract will store counts only.
Like:
yes - 313
no - 131

If user want to change vote than user should sent transaction to personal vote item (you can prepare it in your ddap) and that item will change vote on main contract

You can see simple example here - https://github.com/Tonstarter/simple-vote to understand how it can be work for a lot users without big storage