No experience of a fracture but to me it sounds like you’ve rather overdone it going from almost nothing to 10k steps for 5 days straight.
I would work up to 10k steps gradually with rest days between active days.
Make sure you’re stretching, icing and elevating after exercise. Get in touch with the physio as they are the rehab experts and not the surgeons. Surgeons are often unrealistic about recovery times, giving the best case scenario, everyone is different though.
Ask the physio what other activity you can safely do while recovering, maybe swimming, cycling, upper body work in the gym? If the fracture is on a weight bearing area you won’t want to be pounding it walking. Also pay close attention to your footwear, that it is supportive and cushioned. Maybe you need a knee support for this early stage, ask the physio.
Also, knee injuries can take months and months to resolve and recovery isn’t linear, you’ll have good days and bad days. It’s a marathon not a sprint, and can be mentally draining and frustrating unfortunately!
If it’s still not improving in a month or two, then that may be the time to see the Dr, get another scan etc. He may be busy but it’s his job ultimately to see patients

Good luck