It might be that the question was asked before...

Thank you for your answers. I will try your suggestions on the weekend as for now I don’t have the time to test anything and I will let you all know if I have any updates. Thanks !

So, it was the extruder. The screw that held the bearing in place was a little loose. I also switched to bowden setup to make some more tests and the walls on the test cube are the smoothest I ever had !
Thanks everybody for the suggestions !