The smallest one that will serve the needs of the person(s) who will use the home. Also, the home that will utilize the building materials which can be resourced locally, sustainably and that have low embodied energy.
If building materials have to be shipped long distances by transport, and/or those materials require huge amounts of energy in their mining/manufacture OR they last for a very short time before decaying beyond their use-by date, then they are not efficient in energy use.
If the home is big enough to house double the number of people who WILL use it, then there has been a waste of resources.
It a number of rooms have to be shut off during cold winter months, because it is too expensive to heat them, that again is a waste of resources.
If you go "modern" and insist on using manufactured solar panels for hot water, while there is a usable and sustainable quantity of burnable timber very close, then an efficient wood-burning stove will heat the water more cost-effectively.
Does all the above sound like common sense? I hope so.