Of course its matter
let me show you a few samples. right now I'm searching a keyword on google "Best Vlogging Camera under 100"
Check Attachment for Live Proof of Top 3 Ranking Website. Image Name: Best Vlogging Camera under 100
Now I'm checking the first 3 website page speed.
You can check attachment file for "First Website, Second Website and 3rd Website"
the highest Page Speed Score is 93% "https://vloggingguru.com/best-vlogging-camera-under-100/" Now the question is why 2nd number ranking website has highest Page Speed score but still number second ranking. the reason is there has 200 ranking factors more that can help to rank a website. when you type on google "how many ranking factors does google have" you will get 200 ranking factors that google see in our website. Now we can say maybe the number second website misses something that number one ranking website did it for his website like Schema Markup or something others.
in the end, we can say "yes its matter" but we can never say it only page speed matter its 200 ranking factors also matter.
Thanks