Like has been stated before, The Jedi did not wear normal clothes even for A Galaxy Far Far Away. Jedi were at their core Warrior Monks. Real life examples of Warrior Monks would be The Shaolin, The Knights Templar and Hospitillars. As far as "casual" I don't think there really would be and 1:1 for a Jedi's Robes.
This. But it does seem like SOME Jedi's uniforms, or at least their chosen attire we say, could pass for casual wear in-universe.
For example, Ahsoka's attire looks pretty casual. One the right is from the Clone Wars, when she was actually a Jedi, and the right was when she was no longer a Jedi, but it's not a dissimilar look from her Jedi attire:
And, while it is only fan-art, here's a drawing of something I could totally see Dooku wearing as a Jedi that, while pretty formal, still looks more in-line with today's fashion than your typical Jedi uniform: