My guess? One for All started probably started out as All for One. Instead of passing power around, it began as something that took power from a source, supposedly other humans, and making the quirk and user stronger. However, something happened to the original user and, somehow, the quirk transformed into what it is now. However, a side effect is that those who wielded or, maybe, absorbed by One for All/All for One don't die, or precisely, can't die. Their will, and perhaps, their soul, are all apart of this new hero or, a more accurate term, host. Remember, in order to gain All for One, Deku had to train his body. Now, the idea was that Deku had to train his body in order to handle the stress of All for One, but maybe, it's to handle the stress of these entities within him.
All Might appeared surprised when Deku told him about what he saw during the fight, but it was played for laughs and it appeared as if All Might didn't know anything about it, but after Deku left, he became solemn and it's implied that he does know what going on. Basically, All Might already knows he's half dead. He is literally a shell. He also mentioned that he saw the hallucinations during his youth. Now, to me, this could be taken a couple ways. Either he is lying about just seeing it in his youth, and he still actually sees them, or, one must reach a certain level where these illusions no longer can take control or influence the new vessel. Either way, he is trying to protect Deku from something or wants him to fight it on his own.
This next theory is a bit more out there, but still connected to the first a bit. The luminescent baby isn't the first quirk user, but is somehow connected to the first quirk user who, I assume, had One for All, or at least, some different version of it. I think One for All is the progenitor for all quirks. Someone went around and spread the wealth of power to a good portion of the world. Now, this would mean that, on some level, everyone is connected to One for All, which, I believe, is going to play a big part later on in the series.
So yeah, that's my current theory for One for All.