I have a similar problem with this as I do my with Oden still being alive. For one this would indicate either Lady Toki or Oden all but straight up abandoned Hiyori, which is unlikely. Lady Toki unlike Oden may not necessarily be obligated to save Wano, so I’ll leave it there.

I highly doubt either of Hiyori’s parents would abandon her for 20 years. Maybe if they sent all or half (Momo could get the other half) of the Scabbards to protect Hiyori, but even that wasn’t the case.

Lady Toki knew or at least predicted Kaidou’s and Orochi’s tyranny would last for at least 20 years. Subjugating your daughter through that while you travel through time is absolute insanity and unbefitting of the image Oda has crafted for Lady Toki so far.

Lady Toki used to travel through time often, but she met Oden and decided to finally settle down for once in her life. Then, her great love died. She then sent Momo into the future and told Kawamatsu to protect Hiyori. Then she finally decided to die instead of living on without her love, Oden.

By predicting either Lady Toki or Oden is still alive you either have to shift their characters into asshole of the century or introduce more amnesia nonsense like we already did with O-Lin and Sabo.