If the human mind could swap bodies it would not matter. So only the mind matters.
Your short term memory can only recall about seven things at once. What we focus on at any time may be even fewer. True, we can very quickly retrieve information from long term memory, and we logically infer that an outside world must exist, but we are only directly conscious of seven items or fewer at any moment.
When faced with any difficult question it helps to use Occam's razor. Do you think of "yourself" looking at those seven items in your head? Why? If consciousness can be entirely explained by seven items in short term memory then why do we also need an external observer?
So consciousness is those seven items in short term memory. And you are not a little person in your head thinking of those seven ideas. You are those seven ideas.
Most of our ideas change constantly, and are not important in the long run. E.g. what we eat, how we sleep, using the toilet, etc., are just means to an end. The end itself, the important part, is our survival. But survival of what? If we are just our ideas, what ideas survive from day to day, year to year? Those would be our long term values (and effect on others) matter. So the long term "you" is your values and effect on others, nothing else.
So the real you is shared every time you interact with others. So the real you lives though multiple bodies. You even share some values with animals, plants, and objects like the stars. For example, Pythagoras once recognised a dead friend's personality in a dog's bark.
It might help to remember that time does not flow. So the future "you" is a different person. So there is no advantage in sharing yourself with only that person while excluding all the others.
This sharing of self is the basis of morality. If you live for just your current body then you become selfish, and die when your body dies. But if you see your body as just a tiny part of you, then you focus on others. Then you spread and live on when your body dies.
Occam's razor is an idea popularised by William of Occam (Occam is a towen near London). It refers to shaving off unnecessary parts of a theory (shaving with a razor, get it?). So if two theories could both explain something, and you don't know which is correct, always go for the simpler one.
For why Occam's razor exists, think about why anything exists: everything is relationships between numbers. There are vastly more small relationships than big ones, so if you don't know if the one you see is big or small, choose small.
If it isnt obvious why there are more small relationships, consider choosing between the numbers "8" and "2". The number "8" contains "2" many times over (8 is 2 x 2 x 2, or 2 + 2 + 2 + 2). So there are many times more "2"s than "8"s. Put another way, if you see "2" of something it could be 2 of anything (the items might be made of smaller parts, each made of even smaller parts) but what it is made of does not matter: shave those details off, they are irrelevant to the fact that is is the number "2" .