This pretty much fits with what I have been doing. My entire story was born of a starting scene, and thanks to the way my brain works I needed to start throwing chapters up in order to start a feedback loop and create mental pressure to write.
And that's it. One character's backstory had a detail that gave me the primary antagonist, and I kept building on what went before.
The story is almost entirely about the characters proceeding toward their goals, with occasional extra challenges thrown at them from the outside world. Those goals now include finally dealing with the antagonist, but it's not something they can do immediately. So they continue on with the rest of their plans while building toward being able to deal with the antagonist.
I have a few landmarks to guide my path, but the details of each chapter are discovered as I write them.