To some extent you could address this by using a rifle or shotgun with a very short barrel, but in the US a rifle with a barrel less than 16 inches or a shotgun with a barrel less than 18 inches incurs a $200 tax and requires you to fill out a form, get it signed by your local chief of police, and wait for a few months while the ATF process the paperwork. That's not a big barrier - afterall we do basically the same thing just to own a single-shot .410 with a 24" barrel, but it does put people off when the alternative is to just buy a handgun there and then and save $200. In a few states, short barreled rifles and shotguns are completely banned anyway, and a lot of people have legal concerns about using a federally regulated weapon in a defensive shooting.
More importantly though, you can't fit a shotgun or rifle into your waistband (at least you won't be able to walk with it down there or pull it out quickly!). A lot of the growth in handgun sales in recent years has been as a result of right-to-carry laws and has been concentrated on subcompact and micro-subcompact pistols. I never considered carrying a rifle or shotgun because that's completely impractical. Even if you don't mind open carrying, who wants to carry 6+ lbs of metal around with them all day every day when you can carry a 10 oz handgun that slips into your pocket or waistband.
I do think a shotgun or rifle is an excellent choice if you want something to turtle with i.e. just hide behind your bed waiting for the bad guy to burst in. Both give you the steadiness of a shoulder stock. The rifle can give you a capacity rarely seen in handguns (30 rounds is standard in a rifle, unusual in pistols), while the shotgun is forgiving. People often say that shotguns are more powerful than other weapons but this isn't really the case. A 1 oz slug produces a temporary cavity about 10cm in diameter and a permanent cavity about 2.7cm in diameter. A .45 ACP hollowpoint creates a temporary cavity about 8cm in diameter and a permanent cavity about 2.2cm in diameter. A difference, but not a huge one. 5.56 FMJ creates a temporary cavity about 12cm in diameter (although it has to penetrate 15 inches to produce this - at 10 inches it is 10cm). Slugs penetrate deep, and that's great if you're shooting big game, but humans aren't that big. A .32 ACP can penetrate 18 inches, enough to reach the heart from almost any angle.
But here's 00 buck (similar to SG) at 7 yards (ignore the big hole in the torso):
And here it is at 15 yards:
Instead of a single permanent cavity you get several. This gives you a larger margin for error in your shot placement. A shot centered on the outside of the torso that might have only subjected the vital organs to hydrostatic shock or nothing at all had you been firing a rifle or pistol, might instead penetrate it directly. That's what makes the shotgun so effective.
But if you want to be able to move around - go investigate, try to escape, go get kids from their rooms etc - personally I think the handgun is the better weapon unless you have very wide doorways and corridors. Not that a shotgun or rifle is a bad choice, but consider this: When approaching a doorway or other opening there is 180 degrees of space on the other side, from your point of view at the opening. A bad guy could be in any one of those degrees. With a rifle or shotgun, with a standard doorway, you will have about a 20 degree blindspot to either side, making 40 degrees total. That's one fifth of the space that you can't see. With a handgun this is reduced to about 10 degrees or one eighteenth of the space. (The handgun still has a blindspot because to check the last 10 degrees on either side you need to poke your head out through the doorway, exposing the back of your head to the final 10 degrees on the opposite side).
The other point I'd add is that the over-penetration thing is really overblown. Handguns and shotguns will penetrate multiple interior walls, as will 5.56 and 7.62x39, but none will penetrate exterior brick walls or brick veneer. Unless you're shooting a full size rifle round, which I doubt anybody would want to do, you need to keep in mind that your rounds are like as not going to go right through the house until they hit an exterior wall but, unless they find a window or door, won't be leaving the house to go kill your neighbours. So know where your family are, have a plan, and train, and most any weapon will do a good job.
individual shot penetration is not the whole story, at the range such a shot would be taken inside the average house you are not dealing with individual pellets, rather a discrete mass of lead, the shot column would hit as one lump and will create massive shock trauma, as the range opens up obviously the shot pattern will also open up and the effect will be greatly reduced (I speak from experience here having been "dusted" years ago by a low shot on a clay course that just made it to my position without enough energy to even break the skin) there is also the intimidation factor of a large shotgun pointing at you.as to the options of firing through walls at your target etc I think at that point you are moving well away from the idea of a home defence weapon and more into the realms of armed police operations, and they have different criteria (and access to good legal back-up) Pistols are good if you are legally able to carry them with you where ever you go, which we are not, shotguns can do the job as an emergency self defence weapon, rifles, in our environment are a liability. 
you are quite correct